WTF is going on with Google???

omoxx

Regular Member

Feb 21, 2008

346

18

• Aug 12, 2009

• #1

All of my sites appear in the Webmaster tools as:

"Site not indexed. (Why not?)"

or

"This site may be in violation of Google's quality guidelines. More Details
Pages on your site may not appear in Google search results pages due to violations of the Google webmaster guidelines. Please review our webmaster guidelines and modify your site so that it meets those guidelines. Once your site meets our guidelines, you can request reconsideration and we'll evaluate your site.
Submit a reconsideration request"

In the Crawl errors area, most of them are in:
Restricted by robots.txt

What can I do ?
Help
Thanxxx

 

P

phpfiend

Junior Member

Apr 12, 2009

172

103

• Aug 12, 2009

• #2

Try temporarily removing the robots.txt and going through the steps once again to get them listed in your webmaster tools section.

If your site starts showing up correctly in the webmaster tools after that then you know you need to redesign your robots.txt.

 

omoxx

Regular Member

Feb 21, 2008

346

18

• Aug 12, 2009
• Thread Starter Thread Starter

• #3

Should I remove all non-indexed webs from the list of the webmaster tools, delete robots.txt from all of them and resubmit them again?

The robots.txt I used was this:

User-agent: Googlebot

Disallow: /wp-content/
Disallow: /trackback/
Disallow: /wp-admin/
Disallow: /feed/
Disallow: /archives/
Disallow: /sitemap.xml
Disallow: /index.php
Disallow: /*?
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: */feed/
Disallow: */trackback/
Disallow: /page/
Disallow: /tag/
Disallow: /category/

User-agent: Googlebot-Image
Disallow: /wp-includes/

User-agent: Mediapartners-Google*
Disallow:

User-agent: ia_archiver
Disallow: /

User-agent: duggmirror
Disallow: /

Any idea if it is wrong?
If affirmative, please recommend me a correct one, please!
Thanxxx

 

Trota

Regular Member

Feb 21, 2008

359

123

• Aug 12, 2009

• #4

your robots file is disallowing google bot to crawl any page of your site...

 

T

trizno

Newbie

May 21, 2008

16

8

• Aug 12, 2009

• #5

very funny .. dont just copy paste other works....
keep searching at google... there's lot of answer

User-agent: Googlebot
Disallow: /wp-*


use all-in-one-seo plugins if you want block archives/category/tag

 

omoxx

Regular Member

Feb 21, 2008

346

18

• Aug 12, 2009
• Thread Starter Thread Starter

• #6

What is the use of robots.txt, is it necessary?
When the sitemap is submitted in the status a red X appears?
WTF is that also?