What does your robot.txt file look like?

asiansamurai

Junior Member

Feb 23, 2014

185

35

• Sep 18, 2014

• #1

Hi,

I'm editing my robot.txt file for my PBN's and wanted to ask if you could share how yours look like.

Also how do I block search engines from seeing my plugins?

Do I use this:

Disallow: /plugins/

or do I use this:

Disallow: /wp-content/plugins/

 

RedLable

Regular Member

Feb 16, 2011

244

32

• Sep 19, 2014

• #2

..........

 

SEO Power

Elite Member

Jul 14, 2014

2,642

688

• Sep 19, 2014

• #3

Install Spyder Spanker (it's a plugin) on all your PBN sites if you want to block all link crawlers. There'll be no need to edit your robots.txt file after you do that.

 

asiansamurai

Junior Member

Feb 23, 2014

185

35

• Sep 19, 2014
• Thread Starter Thread Starter

• #4

SEO Power said:

Install Spyder Spanker (it's a plugin) on all your PBN sites if you want to block all link crawlers. There'll be no need to edit your robots.txt file after you do that.

Click to expand...

That's exactly why I want to edit my robot.txt file as using this plugin across all sites leaves a footprint.

Better to be on the safe side after I've spent over $1000 on my PBN.