Country-specific Google Search not working: How to scrape now?

Hey guys,

Ever since Google stopped taking the TLD to show country based results, it is not possible to have different results based on the URL we pick.

I.E.: Gogle.com.br used to show results for Brazil while Google.fr showed results on the french territory.

It was convenient for Scrapebox &/or custom scrapers.

How do you guys do now?

You can try this:


And just change the XX to the country/language you want. For example

https://www.google.fr/search?gl=fr&lr=lang_fr&hl=en&q=cars

Does this display the results you are after?

I also thought I have similar problem but I just dont care. Its based on your IP == location. I went for google.com (click on use "google.com instead" and change ur search engine settings to english etc) in my browser and searched for all the sites with footprint of an single CMS and it returned 40 000 results. I then took some random proxies for scraping and it scraped me about 39,800 so theres almost no loss. Seems if your IP is lets say french - then it will first search for french sites, and then out of better sites it will list english ones. so if you take care of ALL footprints then you will find all results regardless of the proxy IP

Sweetfunny said:

You can try this:

Code:

https://www.google.XX/search?gl=XX&lr=lang_XX&hl=en&q=cars

And just change the XX to the country/language you want. For example

https://www.google.fr/search?gl=fr&lr=lang_fr&hl=en&q=cars

Does this display the results you are after?

Click to expand...

Seems like gl and tld has no effect on the result page. hl is only the interface language. So it seems only lr changes the result pages.

Although it doesn't fully to my question. How am I supposed to check what an american person see versus an english person?

TayaX said:

Seems like gl and tld has no effect on the result page. hl is only the interface language. So it seems only lr changes the result pages.

Although it doesn't fully to my question. How am I supposed to check what an american person see versus an english person?

Click to expand...

use american proxy and then english one

Go here: https://www.google.com/preferences?hl=en&fg=1
Pick the region (scroll down a bit to see the regions) you want to see searches for. Voila, magic.

It was recently updated

https://www.mariehaynes.com/google-geolocation-change-how-to-see-serps-from-another-country/

As noted in different posts and ways above google made some changes at the end of October.

The google country does still have an effect, but the geo local of your IP is going to skew your results. Somewhere from somewhat to dramatically, depending on what you are looking for. If you are on my mailing list I sent out a mail about this last week with details and how to do it now and a proxy provider etc..

Basically for the ideal results, like you used to get, you need an ip(s) from the country that you want results from.