Difficulty with the use of the Scrapebox

Ranoo

Newbie
Joined
Apr 25, 2023
Messages
6
Reaction score
0
I tested the google engine and found that the next page marker was not found, but i don't know how to fix this problem

1682494186899.png
 
I tested the google engine and found that the next page marker was not found, but i don't know how to fix this problem

View attachment 252360
Go to settings >> harvester engine configuration >> import >> download default engines.

If that fails then

Go to settings >> harvester engine configuration



Then click google

Then replace the “Marker For Next Page” box, replace it with this line of code (means delete the existing code and paste this new code in. In this 1 box only):



<span style="display:block;margin-left:53px">|<img border="0" src="nav_next.gif"|src="nav_next_2.gif" width="100"|<br><strong>Next</strong>|filter=0">Next|<span>&gt;</span>|aria-label="More results"



Give the Display Name a new name.



Click add as new engine



Then when you go to harvest select that newly named engine.
 
Go to settings >> harvester engine configuration >> import >> download default engines.

If that fails then

Go to settings >> harvester engine configuration



Then click google

Then replace the “Marker For Next Page” box, replace it with this line of code (means delete the existing code and paste this new code in. In this 1 box only):



<span style="display:block;margin-left:53px">|<img border="0" src="nav_next.gif"|src="nav_next_2.gif" width="100"|<br><strong>Next</strong>|filter=0">Next|<span>&gt;</span>|aria-label="More results"



Give the Display Name a new name.



Click add as new engine



Then when you go to harvest select that newly named engine.
Thanks to loopline for the answer, it works!!
But many keywords have no harvested results, is this harvester status working properly?

1682560415419.png
 
Thanks to loopline for the answer, it works!!
But many keywords have no harvested results, is this harvester status working properly?

View attachment 252487
Yes it works, but it will only attempt a keyword up to the number of retries you have set.

So if you go to settings > connections timeouts and other settings >> more harvester options >> proxy retries

set that proxy retries box to the max. Because scrapebox will try however many quantity of proxies you have set there and then it will skip the keyword.

glad its working!
 
Yes it works, but it will only attempt a keyword up to the number of retries you have set.

So if you go to settings > connections timeouts and other settings >> more harvester options >> proxy retries

set that proxy retries box to the max. Because scrapebox will try however many quantity of proxies you have set there and then it will skip the keyword.

glad its working!
Hi @loopline, thanks for your help.
I set the maximum of proxy retries according to your method, and tested it for a while, but some keywords still have no harvester results.

1682736856225.png1682736870431.png1682736966860.png
 
If keywords are being skipped then you have a lot of dead proxies. Because lets say you have proxy retries set to 10, then scrapebox will try 10 proxies before skipping the keyword. Plus your 429 test is a blocked ip.

but you just need to wait 48 hours and see if the proxies become unblocked, assuming they are paid proxies and if not get new proxies. Else if they are public proxies, then Im impressed you are getting as good of resutls as you are.

you also can try using the detailed harvester with a delay to keep the proxies from getting blocked. But bear in mind the detailed harvester will retry indefinitely, which means if you go to fast you will eventually burn out all your proxies as blocked.
 
If keywords are being skipped then you have a lot of dead proxies. Because lets say you have proxy retries set to 10, then scrapebox will try 10 proxies before skipping the keyword. Plus your 429 test is a blocked ip.

but you just need to wait 48 hours and see if the proxies become unblocked, assuming they are paid proxies and if not get new proxies. Else if they are public proxies, then Im impressed you are getting as good of resutls as you are.

you also can try using the detailed harvester with a delay to keep the proxies from getting blocked. But bear in mind the detailed harvester will retry indefinitely, which means if you go to fast you will eventually burn out all your proxies as blocked.
Okay, I'll wait 48 hours. and I bought this from myprivateproxy MPP Web Scraping 10 - Private proxies for Scraping.
Or can I ask you to recommend a more stable proxy?
1682994498845.png
 
Back
Top