Where is robots.txt in wordpress?

loclhero · Nov 20, 2011

I can't find this sucker. G00g is indexing the images on my site which is pulling too much resources and I'm getting bogus site info on my awstats (see tons of searches for a kw but it's coming from g00g images).

I can't find the file and want to disallow my images file.

wannabie · Nov 20, 2011

It might not have one, go into your host and upload a file called robots.txt (same folder as your wordpress) then disallow what you want

You can use noindex,nofollow meta on the pages if you cant get into your host?

hit me up on skype and ill talk you through it if you want

huubert · Nov 20, 2011

It isn't there by default AFAIK. Just create one and upload it to your public_html root.

blackhit · Nov 20, 2011

It isn't part of the WP install. You'll have to upload it manually to the root of the domain (assuming your blog is in the root).

This is the one I use. You just add your images folder to it with the disallow command.

Code:

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /feed
Disallow: /comments
Disallow: /category/*/*
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /*?*
Disallow: /*?

# Internet Archiver Wayback Machine
User-agent: ia_archiver
Disallow: /

# digg mirror
User-agent: duggmirror
Disallow: /

Here's the info for getting your images out of the index.

Code:

http://www.google.com/support/webmasters/bin/answer.py?answer=35308

jazzc · Nov 20, 2011

Also, I would recommend Yoast WordPress SEO plugin.

It has an Indexation section that lets you finetune lots of things one would have easily missed.

Where is robots.txt in wordpress?

loclhero

Supreme Member

wannabie

Elite Member

huubert

Registered Member

blackhit

Elite Member

jazzc

Elite Member

Main Menu

Marketplace

Making Money

BlackHatWorld

Other