Google PAA & PASF Scraping

[Alexander301]

Hi folks,
Anyone have a reliable method to scrape the people also ask and people also search for queries from Google? I have a python script that does that but it times out after 80-90 ish queries and I have to wait for it to reset.

I would like to either have it scrapped directly in python one by one but reliably for hours or another method to just get that data say for 500-1000 keywords as needed.

 

[Zher0]

I believe Google recently updated their scraping security, most of the data center proxies i use now are blocked. You might want to try using 4g mobile proxies?

 

[Algo]

Have you tried to use a service like ScraperApi or Rocket scrape...

 

[HustleTong]

SEO Minion does this if i'm not mistaken

 

[hideath]

You have to use proxies whenever you're using your own script or any other script/program,
You'll hit a limit/captcha after a while, you have to go with Google proxies & multi-thread to get things done at a scale,

The limit is 2 minutes per Google search, which makes 30 Google searches per IP per hour,
So if you need to scrape 1k keywords per day, you have to use at least 2-3 proxies,

You can also use https://serper.dev/,
They offer a really good price, and you'll get the data much faster

 

[Zher0]

You have to use proxies whenever you're using your own script or any other script/program,
You'll hit a limit/captcha after a while, you have to go with Google proxies & multi-thread to get things done at a scale,

The limit is 2 minutes per Google search, which makes 30 Google searches per IP per hour,
So if you need to scrape 1k keywords per day, you have to use at least 2-3 proxies,

You can also use https://serper.dev/,
They offer a really good price, and you'll get the data much faster

Funny enough, I was doing my own research and just started using them as well (i'm not affiliated). Haven't had any issues with their trial! Scrapebox no longer looks to be working post Google javascript update

 

[Alexander301]

You have to use proxies whenever you're using your own script or any other script/program,
You'll hit a limit/captcha after a while, you have to go with Google proxies & multi-thread to get things done at a scale,

The limit is 2 minutes per Google search, which makes 30 Google searches per IP per hour,
So if you need to scrape 1k keywords per day, you have to use at least 2-3 proxies,

You can also use https://serper.dev/,
They offer a really good price, and you'll get the data much faster

I checked it out and the pricing seems decent. I was using my own script but serper seems like a better alternative. Thanks

 

[hideath]

Funny enough, I was doing my own research and just started using them as well (i'm not affiliated). Haven't had any issues with their trial! Scrapebox no longer looks to be working post Google javascript update

I checked it out and the pricing seems decent. I was using my own script but serper seems like a better alternative. Thanks

Glad this helps,
It would be more cost-effective for a shorter/minimal scraping in fact,
For the longer term/larger scale, you can get cheap proxies from newipnow (they have a thread in bhw) along with a custom script (I'm not aff)

 

[Techpreneur]

Hi folks,
Anyone have a reliable method to scrape the people also ask and people also search for queries from Google? I have a python script that does that but it times out after 80-90 ish queries and I have to wait for it to reset.

I would like to either have it scrapped directly in python one by one but reliably for hours or another method to just get that data say for 500-1000 keywords as needed.

I developed an advanced Python script to scrape PAA questions a year ago, but I'm not sure if it still works. I think I could help you. Give me more details.