Robots.txt for PBN

rakeem · Feb 25, 2015

Hi dudes,

i want no plugin for hiding my PBN. Please post the codes for robots.txt or .htaccess to secure PBN.

I would appreciate it if you dont post any plugins for this.

Thanks... Cheers.

watsonovedades · Feb 25, 2015

you mean you want to hide your links and avoid the google spiders to crawl them?

nycdude · Feb 26, 2015

rakeem said:
Hi dudes,

i want no plugin for hiding my PBN. Please post the codes for robots.txt or .htaccess to secure PBN.

I would appreciate it if you dont post any plugins for this.

Thanks... Cheers.

If you mean to keep spiders of your page it's on a robots.txt file with this:

Code:

User-agent:*

Disallow: /

Upload that to the path you want to keep the spiders away ... if that's what you want to do that it.

jollyman · Feb 26, 2015

Check this out http://www.robotstxt.org/robotstxt.html I hope this will help you.

Lalalaenhund · Feb 26, 2015

nycdude said:
If you mean to keep spiders of your page it's on a robots.txt file with this:

Code:

User-agent:* Disallow: /

Upload that to the path you want to keep the spiders away ... if that's what you want to do that it.

Correct me if I'm wrong but if this is for a PBN, I think you need to add a line to allow Google spiders, otherwise your network will be kind of useless.

rakeem · Feb 26, 2015

i want to block all the unnessasary bots to hide my PBN. I know i can use plungins like spider spanker pro, etc. but as i said i dont want any plugin.

Cheers...

Aty · Feb 26, 2015

Try this.

accelerator_dd · Feb 26, 2015

Or just:

User-agent: *
Disallow: /
User-agent: Googlebot
Allow: /
User-agent: Slurp
Allow: /
User-Agent: msnbot
Allow: /
User-Agent: bingbot
Allow: /

You are better off whitelisting a few than blacklisting a bunch.

oscilay · Feb 26, 2015

Don't do that for every website as that s a footprint in case of manual inspection from big G.

rakeem · Feb 26, 2015

accelerator_dd said:
Or just:

You are better off whitelisting a few than blacklisting a bunch.

i guess you are right

rakeem · Feb 26, 2015

oscilay said:
Don't do that for every website as that s a footprint in case of manual inspection from big G.

that means not to hide the PBN. makes no sense.

oscilay · Feb 26, 2015

rakeem said:
that means not to hide the PBN. makes no sense.

Ok. Don`t post here when your PBN will be deindexed.

Robots.txt for PBN

rakeem

Junior Member

watsonovedades

Supreme Member

nycdude

Power Member

jollyman

Elite Member

Lalalaenhund

Power Member

rakeem

Junior Member

Aty

Elite Member

accelerator_dd

Elite Member

oscilay

Power Member

rakeem

Junior Member

rakeem

Junior Member

oscilay

Power Member

Main Menu

Marketplace

Making Money

BlackHat World