ScrapeBox Automator getting stuck after sometime

[TravelToday]

I am running latest scrapebox v2 with Automator to scrape targets for my gsa search engine ranker (GSA)





Once i run Scrapebox harvester with automator it starts to scrape then after some time its getting stuck. That way i am wasting lot of resources as SB got stucked midnight. Also it doesn't even save my remaining keywords (guess there is no option to save remain footprits + kwrds , We just can place remain kwrds to keyword memo since its getting stuck next time it won't appear.


Would like to mention and specially expecting answer from master of SB @loopline

Also what you guys averaged URLS per second. Mine is around <300 links which is pretty low.


Bing is broken. (even i updated engines from server)
Google is no way to scrape - This is why i purchased a verified list from few providers as seems scraping is very hard



Pardon my English as its my 3rd language

 

[YesSir532]

Use lots of proxies. It will really gets blocked if you only have few.

 

[TravelToday]

Use lots of proxies. It will really gets blocked if you only have few.

I am using private proxies 100 which is passed for bing. I don't think thats the issue. Seems Scrapebox cant handle that much of footprints. I just loaded half of it. Will update

 

[YesSir532]

I am using private proxies 100 which is passed for bing. I don't think thats the issue. Seems Scrapebox cant handle that much of footprints. I just loaded half of it. Will update

Hope so. Keep us posted

 

[TravelToday]

This time only tried to scrape articles footprint and again SB got stucked after some time and memory using alomost 8gb. My Server is 32GB btw

i just uesd article footprints to scrape with 5 keywords :/

@loopline sir i really need help from you.

 

[loopline]

Your on beta 4, if you update to beta 7, or whatever the latest is, by the time you read this, try that.

 

[TravelToday]

Your on beta 4, if you update to beta 7, or whatever the latest is, by the time you read this, try that.

Wow! I just forgot that...thats why you are a legend of SB!

I updated and now its 400 urls per sec on yahoo.

But still not sure about getting stuck issue after some time. I will update as it helps others too.

May i know what you mostly scrape yahoo or bing ? with how many private proxies and thread.

I am looking to buy GSA proxies scraper will see.

Already subbed to all of you channels and was a loopline customer

 

[loopline]

Wow! I just forgot that...thats why you are a legend of SB!

I updated and now its 400 urls per sec on yahoo.

But still not sure about getting stuck issue after some time. I will update as it helps others too.

May i know what you mostly scrape yahoo or bing ? with how many private proxies and thread.

I am looking to buy GSA proxies scraper will see.

Already subbed to all of you channels and was a loopline customer

I mostly scrape google to be honest, I like the accuracy.

Bing is my 2nd choice, I basically never scrape yahoo.

Thats me though, thats for my needs, Im not saying you should or should not scrape any engine, because your needs will be different.


I suspect, unless yahoo gets some sort of AI in the next couple years, that it will sort of die as a search engine.

My bet would also be that once chat gpt gets more heavily engineered into Bing and bing can use it to really understand what queries a user wants and whats actually relevant, that bings accuracy will go way up.

However the only thing I really know is that over the next coming years, as AI takes a more prominent foot hold in everything, that many things will change, so stay on your toes!


As for proxies, its about speed. The more proxies you have, that work with the engine your using, the faster you can go. So find a threshold that works for you.

Sometimes if Im just doing a small scrape, I use a delay and skip proxies entirely on google. Other times I load up 50 or 100 or whatever and go. But I also have archives of data from various sources and over the years, so very often I use internal data and just keep it refreshed with external data these days. So point is your need may be different then mine.

 

[TravelToday]

I mostly scrape google to be honest, I like the accuracy.

Bing is my 2nd choice, I basically never scrape yahoo.

Thats me though, thats for my needs, Im not saying you should or should not scrape any engine, because your needs will be different.


I suspect, unless yahoo gets some sort of AI in the next couple years, that it will sort of die as a search engine.

My bet would also be that once chat gpt gets more heavily engineered into Bing and bing can use it to really understand what queries a user wants and whats actually relevant, that bings accuracy will go way up.

However the only thing I really know is that over the next coming years, as AI takes a more prominent foot hold in everything, that many things will change, so stay on your toes!


As for proxies, its about speed. The more proxies you have, that work with the engine your using, the faster you can go. So find a threshold that works for you.

Sometimes if Im just doing a small scrape, I use a delay and skip proxies entirely on google. Other times I load up 50 or 100 or whatever and go. But I also have archives of data from various sources and over the years, so very often I use internal data and just keep it refreshed with external data these days. So point is your need may be different then mine.

Worth every word you said!


I can't even think of scraping google. Even bing is hard for me.. <100 links per sec.. yahoo getting around 300 + .

While scraping i am now subscribed to 2 link list providers and its worth for my money and i am getting decent stable vpm 30+ .

So its hassle free now.