You seemingly still think I use web 2.0s a lot, not sure why you think that since I last spoke on that front at least 2 years ago. I have a network that has over 1000 sites in it now.
However, your links can be found, rest assured. I could probably find all of them without even going to any 3rd party crawler at all just by footprints on your network. I've done this before, many times.
A footprint on your topic for your site, a url from your site + this ("user-agent" "robots.txt" "AhrefsBot") will generate every single backlink you have. And this is in google....
Who are you trying to protect yourself from exactly? Because any semi decent SEOer will figure out every link they want to figure out. I do this on a regular basis already. Even if you're really smart and don't go through robots.txt (which isn't entirely effective if you don't) I'm positive I could locate all sites in your network.
If you really really care about your PBN, design a plugin that removes all external links when certain bots show up.
I just searched a client site of mine with:
site.com + "ahrefsbot" "robots.txt" "user-agent"
Not a single result in Google, while that site has 20 links from my network.
Obvious this can work to a certain degree but only when the URL is used as anchor text and then you don't even need to add the ahrefsbot robots.txt and user-agent stuff.
Even when I take it one step further and type in my main keyword that I like to rank for (which is multiple times used) only 5 results show up and none of those sites belong to my network.
I have a good idea why this doesn't work, as this string of robots, user-agent- ahrefs returns a robots.txt page obviously and that page does not contain my money site URL or money keywords so it would be impossible to find my network that way.
For your pleasure I even took it one step further.
I copied the first 5 lines of my robots.txt in Google and put it within quotes, again not a single one of my network sites show up.
Now when I type in only the URL of that specific site in Google I do see 3 of my network sites show up. Now obvious I could block the user-agents from tons of web browsers but that wouldn't change a single thing as then they could only not visit my private network site but it would still show up in Google.
To get back at your original statement as that is what I commented on (instead of side railing it to that you can find PBN sites in Google:
"
In fact, no matter how you try (robots.txt, htaccess, etc) they will often still ignore it."
NOPE they will NEVER ignore it when you set it up properly.
