Can you scrape 100 results (with scrapebox) now num=100 is now allowed?

Just getting ready to start a scrape, looked at the harvester settings and I can see that num=100 is being used, obviously now this is obsolete.



Does scrapebox still work the way it used to, are there any other additonal settings I should consider before I perform the scrape?

I have 55 proxies to scrape 915 Google keywords but I will use a delay of 30 seconds before scraping each keyword and leave it running throughout the day.

@loopline any help here would be appreciated mate

You want to be using the newer harvester in Scrapebox (tick 'Beta' next to 'Custom Harvester'), which uses Chrome via Python. Make sure you've updated to the latest version, which is V2.1.0.57. However, that is still limited to 10 results per page just like everyone else. You can scrape more results by scraping additional pages, but scraping anything past the initial page is where most automated tools start to struggle and hit bans/captchas.

Steptoe said:

You want to be using the newer harvester in Scrapebox (tick 'Beta' next to 'Custom Harvester'), which uses Chrome via Python. Make sure you've updated to the latest version, which is V2.1.0.57. However, that is still limited to 10 results per page just like everyone else. You can scrape more results by scraping additional pages, but scraping anything past the initial page is where most automated tools start to struggle and hit bans/captchas.

Click to expand...

@Steptoe got it, beta harvester.

Just put in quantity of results and it will manage pages automatically.

obviously google and other engines limit results to 200 or LESS and less is the keyword here.

You can merge other keywords into your keywords, so if you have 915 keywords and they are location based you could merge on city/state or post codes etc.. Get more variety of results and remove duplicates when done.

Not exactly the answer you want but try detailed harvester. It’s slow. But you’ll get all the results there can be. Or at least that’s how it used to work.

Also as @loopline said use the merge feature with tons of keywords or a list like:

aa
ab
ac
And so on until you get to zz

Lots of ways to scrape. Even before Google took the 100 results at a time feature we used keywords or numbers or letters behind the keyword to scrape EVERYTHING.

Then dedup the list!