1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

WTF is going on with Google???

Discussion in 'Blogging' started by omoxx, Aug 12, 2009.

  1. omoxx

    omoxx Regular Member

    Joined:
    Feb 21, 2008
    Messages:
    346
    Likes Received:
    18
    All of my sites appear in the Webmaster tools as:

    "Site not indexed. (Why not?)"

    or

    "This site may be in violation of Google's quality guidelines. More Details
    Pages on your site may not appear in Google search results pages due to violations of the Google webmaster guidelines. Please review our webmaster guidelines and modify your site so that it meets those guidelines. Once your site meets our guidelines, you can request reconsideration and we'll evaluate your site.
    Submit a reconsideration request"


    In the Crawl errors area, most of them are in:
    Restricted by robots.txt

    What can I do ?
    Help
    Thanxxx
     
  2. phpfiend

    phpfiend Junior Member

    Joined:
    Apr 12, 2009
    Messages:
    172
    Likes Received:
    102
    Occupation:
    Web Dev and IM
    Home Page:
    Try temporarily removing the robots.txt and going through the steps once again to get them listed in your webmaster tools section.

    If your site starts showing up correctly in the webmaster tools after that then you know you need to redesign your robots.txt.
     
  3. omoxx

    omoxx Regular Member

    Joined:
    Feb 21, 2008
    Messages:
    346
    Likes Received:
    18
    Should I remove all non-indexed webs from the list of the webmaster tools, delete robots.txt from all of them and resubmit them again?

    The robots.txt I used was this:


    User-agent: Googlebot

    Disallow: /wp-content/
    Disallow: /trackback/
    Disallow: /wp-admin/
    Disallow: /feed/
    Disallow: /archives/
    Disallow: /sitemap.xml
    Disallow: /index.php
    Disallow: /*?
    Disallow: /*.php$
    Disallow: /*.js$
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: */feed/
    Disallow: */trackback/
    Disallow: /page/
    Disallow: /tag/
    Disallow: /category/

    User-agent: Googlebot-Image
    Disallow: /wp-includes/

    User-agent: Mediapartners-Google*
    Disallow:

    User-agent: ia_archiver
    Disallow: /

    User-agent: duggmirror
    Disallow: /


    Any idea if it is wrong?
    If affirmative, please recommend me a correct one, please!
    Thanxxx
     
  4. Trota

    Trota Regular Member

    Joined:
    Feb 21, 2008
    Messages:
    356
    Likes Received:
    121
    your robots file is disallowing google bot to crawl any page of your site...
     
  5. trizno

    trizno Newbie

    Joined:
    May 21, 2008
    Messages:
    16
    Likes Received:
    8
    very funny :p .. dont just copy paste other works....
    keep searching at google... there's lot of answer

    User-agent: Googlebot
    Disallow: /wp-*


    use all-in-one-seo plugins if you want block archives/category/tag
     
  6. omoxx

    omoxx Regular Member

    Joined:
    Feb 21, 2008
    Messages:
    346
    Likes Received:
    18
    What is the use of robots.txt, is it necessary?
    When the sitemap is submitted in the status a red X appears?
    WTF is that also?
    :eek: