Adwords clocking logs and possible fake referers

[affiliatex]

Hi BHW,

so I've been using a cloaking script and running adwords for about a month. I'm not currently redirecting or doing and funny business yet but running the ads through the script to log all details and gather info on 3rd party companies looking for my ads and google bots etc (yes it costs $$$ but the info gathered is priceless)....

In my logs I've seen several instances in my logs where keyword data seems to be indiscriminately placed into the referer string. From what I understand, google no longer allows keyword data and has encrypted all searches to prevent this. Hence why all the Seo's are pissed because of (not provided) in analytics.

Here's some logs (edited slightly for anonymity)

5/21/2014 12:42 PM | UA: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/537.36 | IP: 68.13.98.151
HOST: ip68-13-98-151.om.om.cox.net | REFERRER: http://www.google.com/aclk?sa=L&ai=C5YA7Z-V8Z6HsB6jA-QOh44DICrGHou8E4bHTp5kBgcqzkQgIABABKAJQ9MC8iAJgydbNiqyktBTIAQGqBB9P0F6Ul8M3Im8B15-7EdrEips5M8Yl0sUWDb5zsYEdugUTCJdfDFEQn5vIvb4CFZBeiAod23AA78oFAIAH-Y3hL5AHAw&ei=Z-V8U9mABJC9oQTb4YH4Dg&sig=AOD64_2jjEEgLWDaQedqC0eBo8plV1QEuw&rct=j&q=&sqi=2&ved=0CCcQ0Qw&adurl=http://goo.gl/12345

The log above seems like a normal log from a normal user. No issues here. However I am seeing the keyword insertion anomaly and data being appended to my destination URL in the log highlighted below (I have Auto Tagging turned off in adwords).

5/21/2014 12:49 PM | UA: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36 | IP: 98.202.187.170
HOST: c-98-202-187-170.hsd1.ut.comcast.net | REFERRER: http://www.google.com/aclk?sa=L&ai=CfGd8U67bK8rU-QOdnILwAbGHou8E4bHTp5kBgcqzkQAggHJBKANQ9MC8iAJgyYaAgOikjBjIAQGqBB9P0EU95VQ3bZHSeRU2l5k4ruWufmiK1zRNPdiz6E9uugUTCP641ObJvb4CFUU6iAod5FMA-8oFAIAH-Y3hL4gHAZAHAg&rct=j&q=KEYWORD&ei=Eed8U77sJ8X0oATkp4HYDw&sig=AOD64_3-kh_3dRXdtu1qCMiCK_0lgeBhIA&sqi=2&ved=OfH0Qw&adurl=http://goo.gl/12345&cad=rjt&q=keyword

So the question is, why am I seeing this "anomaly". Is this someone faking the referer to appear as a real google adwords click to "reveal" my real page? Is it google ad teams super secret system to catch cloakers? or a 3rd party google ad monitoring company? etc...

This is an obvious footprint of sorts and I'm calling BS on it being a real search because if those obvious keyword variables were present at all, webmasters could easily capture their precious keyword data as it is obviously un-encrypted. Please correct me if i'm wrong.

I'm looking for a little insight and opinions of what this is or to confirm or dismiss my suspicions.

 

[mrblackjack]

Both hits are legit, and Google doesnt hide the keyword data when it comes to traffic from Adwords (though, lately, they began doing it too), but hiding it when traffic is organic. And, you have a long way to pass if you want to catch on Google IPs (not the bot IPs, but the 3rd party and manuals), as you need to run campaigns all over the world and for numerous niches for a period of time. A rule of thumb to help you here is, you need data, and huge to start working with.

Also, relying on blocked IPs wont help you, as Google uses manual reviewers to check your campaigns who use dynamic IPs, which means, IPs you cannot really block as real users pass through them too. So, my tip for you, dont rely on IPs (nor other headers info like: useragent, host, referrer etc), as they will get you half the way, but consider other ways (this is why you need lots of data)

Good Luck

 

[affiliatex]

Both hits are legit, and Google doesnt hide the keyword data when it comes to traffic from Adwords (though, lately, they began doing it too), but hiding it when traffic is organic. And, you have a long way to pass if you want to catch on Google IPs (not the bot IPs, but the 3rd party and manuals), as you need to run campaigns all over the world and for numerous niches for a period of time. A rule of thumb to help you here is, you need data, and huge to start working with.

Also, relying on blocked IPs wont help you, as Google uses manual reviewers to check your campaigns who use dynamic IPs, which means, IPs you cannot really block as real users pass through them too. So, my tip for you, dont rely on IPs (nor other headers info like: useragent, host, referrer etc), as they will get you half the way, but consider other ways (this is why you need lots of data)

Good Luck

Thanks for the reply mrblackjack,

I've seen other odd behavior as well (i'll get into it later in the thread). You're right, the encrypted search is for organic and not adwords. I still am a bit baffled on why adwords would be auto tagging my destination URLs even though I have it turned off.

adurl=http://goo.gl/12345&rct=j&q=keyword

No analytics code within my redirect pages either. This is why i'm skeptical of a legit user or not. As far as I can tell, that appendage is only used for autotagging for analytics purposes only. I've also found instances of where I've found where my whole redirect path has been recorded and then parts of it used as a referer to try to access my lander. EX: redirect1.com, redirect2.com, redirect3.com, lander.com they would use REFERER: redirect2.com. Seems spoofed to me. So the questions begs, why is this data sometimes there and then not? I can't replicate the result even if I click my own ads with different browsers, ip's etc.

Also, wouldn't this go a long way in redirecting the big G?

http://bgp.he.net/search?search%5Bsearch%5D=google&commit=Search

 

[Atomic76]

A couple more things I've noticed over the years with AdWords...

- One surefire way to trigger their "bot" to spider your landing pages repeatedly is to simply add new keywords /ads / landing page URL's. Every time you make changes like this to your account, they send their bot through each keyword, to each possible ad and corresponding landing page that you have configured in AdWords. This makes sense, since they need to evaluate the landing page quality, and it's relevancy to your ads and keywords. I don't recall if Google Analytics automatically hides this from it's reports, but at previous places I worked - which had their own internal reporting systems in place, we would always see a spike in logged "visits" (as they were defined in the company's reporting tools) that roughly equaled the number of keywords I added to the new campaign(s).

- If you have Google Search Partners enabled, and not just Google Search itself, they all seem to label their URL parameters a little different. So if you have some sort of rules/formulas to "scrape" off different URL parameters from your data, you could be missing some of it from the Search Partners traffic specifically. For example, Google might label the keyword parameter something like "keyword=", but some of their search partners might label it as "kwd=" or "k=", and it's the same thing with the "query=" parameter, which might be something like "q=".

Also, my apologies if you are already aware of this, but keep in mind that Keyword and Query are two different things. The Query is what the user actually typed in the search box, the Keyword is what keyword in your account the search query was mapped to. So if the user did a search on "cheap granite countertops florida" and you were bidding on the Keyword phrase "granite countertops" on Broad Match, your ad may have been shown to that user and they could have clicked on it.

Likewise, the Referrer URL and the Landing Page URL are two different things. The Referrer URL may be something like Google or AOL (where they are coming *from*) with a bunch of useful parameters you can parse, but the Landing Page URL would be what you have set up in AdWords to send the user *to*, which will have it's own set of useful parameters depending on how you have it set up.