Is My Robots File Stuffed?

mohnikinsiz

Regular Member
Joined
Dec 17, 2011
Messages
334
Reaction score
114
I'm a bit worried that my robot.txt file is stuffed up. I'm running thesis theme on my site and this is what it looks like.

Allow: /
Disallow: ~
Sitemap: http://www.domain.net/sitemap.xml
User-Agent: MJ12bot
User-agent: Googlebot-Mobile
sitemap:http://cdn.attracta.com/sitemap/12x6x4x.xml.gz (it says that this xml file has no style information associated with it)


I know that Disallow: / means to block everything but what does Disallow: ~ mean?

Appreciate any help I can get. Thanks a lot guys!
 
What do you want to disallow? For example if you don't want the bots to access the cgi-bin folder than put this Disallow: /cgi-bin/ I believe your disallow is incorrect in your robot.txt file.
 
Back
Top