1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

GWT "Fetch as Googlebot" VS ordinary Gooblebot

Discussion in 'Cloaking and Content Generators' started by SpamEngineOptimizer, Aug 26, 2011.

  1. SpamEngineOptimizer

    SpamEngineOptimizer Newbie

    Joined:
    Aug 4, 2011
    Messages:
    2
    Likes Received:
    0
    Hi, I need to recognize if website is viewed by ordinary Googlebot crawling the web or by Google Webmaster Tool's "Fetch as Googlebot" function.

    There is no difference in user agent etc., my first idea was to determine GWT FaG IP's but it's using the same addresses as normal Googlebot.

    So... Are there any schemas of crawling the web which Googlebot uses? Have you any ideas how recognize if it is regular crawling or just "test visit" by GWT?

    I need it to make a 3rd party script counting Googlebots visits excluding those from GWT. Thx for any ideas.
     
  2. gavin8492

    gavin8492 Newbie

    Joined:
    Aug 18, 2011
    Messages:
    24
    Likes Received:
    1
    not sure mate sorry
     
  3. machines-in-motion

    machines-in-motion Junior Member

    Joined:
    Aug 11, 2010
    Messages:
    129
    Likes Received:
    46
    Occupation:
    Engineer
    Location:
    Here and there
    Have you tried running a packet sniffer on the server and checking for differences between googlebot and "fetch as googlebot"?