1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[Robots.txt] What is the correct User agent for Adsense Crawl

Discussion in 'Black Hat SEO' started by bluemonster, Mar 21, 2013.

  1. bluemonster

    bluemonster Junior Member

    Joined:
    Jun 13, 2011
    Messages:
    106
    Likes Received:
    13
    Location:
    Adent.io
    Home Page:
    Happen to face Google adsense crawl error, found the fix.

    It says, to make adsense to crawl your pages and show targetted ads, add these lines on top of robots.txt.

    Code:
    User-agent: Mediapartners-Google 
    Disallow:
    But my robots.txt file has already had this lines

    Code:
    User-agent: Mediapartners-Google*
    Allow: /
    
    User-agent: Googlebot-Image
    Allow: /wp-content/uploads/
    
    User-agent: Adsbot-Google
    Allow: /
     
    User-agent: Googlebot-Mobile
    Allow: /
    Someone explain me what these lines refer to and what is the difference between both.

    Which is the right one? Do I need to make any changes or just leave like that?