Y T Nuke  
Results 1 to 3 of 3
Hi, I need to recognize if website is viewed by ordinary Googlebot crawling the web ...
  1. #1
    Join Date
    Aug 2011
    Posts
    2
    Reputation
    10
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default GWT "Fetch as Googlebot" VS ordinary Gooblebot

    Hi, I need to recognize if website is viewed by ordinary Googlebot crawling the web or by Google Webmaster Tool's "Fetch as Googlebot" function.

    There is no difference in user agent etc., my first idea was to determine GWT FaG IP's but it's using the same addresses as normal Googlebot.

    So... Are there any schemas of crawling the web which Googlebot uses? Have you any ideas how recognize if it is regular crawling or just "test visit" by GWT?

    I need it to make a 3rd party script counting Googlebots visits excluding those from GWT. Thx for any ideas.

  2. #2
    Join Date
    Aug 2010
    Location
    Here and there
    Posts
    127
    Reputation
    28
    Thanks
    44
    Thanked 44 Times in 21 Posts

    Default Re: GWT "Fetch as Googlebot" VS ordinary Gooblebot

    Have you tried running a packet sniffer on the server and checking for differences between googlebot and "fetch as googlebot"?
    Python? Check. Caffeine? Check. Sanity? Eh. <Cracks knuckles>

Dot Gov Backlinks Sale


Smarter Submit

Similar Threads

  1. Replies: 19
    Last Post: 08-28-2011, 02:13 AM
  2. All my web2.0s indexed but no link in GWT
    By Scipio00 in forum White Hat SEO
    Replies: 5
    Last Post: 08-15-2011, 05:16 AM
  3. VPS vs ordinary hosting
    By manjack in forum Web Hosting
    Replies: 0
    Last Post: 05-18-2011, 10:01 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
  SEnukeX SEO Software
Proudly Powered by Hostwinds.com Web Hosting Click Here For Exclusive BHW Discounts!

Cheap Web Hosting


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76