GWT "Fetch as Googlebot" VS ordinary Gooblebot

Joined
Aug 4, 2011
Messages
2
Reaction score
0
Hi, I need to recognize if website is viewed by ordinary Googlebot crawling the web or by Google Webmaster Tool's "Fetch as Googlebot" function.

There is no difference in user agent etc., my first idea was to determine GWT FaG IP's but it's using the same addresses as normal Googlebot.

So... Are there any schemas of crawling the web which Googlebot uses? Have you any ideas how recognize if it is regular crawling or just "test visit" by GWT?

I need it to make a 3rd party script counting Googlebots visits excluding those from GWT. Thx for any ideas.
 
Have you tried running a packet sniffer on the server and checking for differences between googlebot and "fetch as googlebot"?
 
Back
Top