Faster url/list scraping? PLS HELP

fullwall3t · Sep 12, 2015

Hello,
is there any way to set more efectivly GSA SER to scrape URL, because i have scrapebox and that is million times faster, but problem is that it freezes in the middle of the process and if i add to many KWS and want to stop harvesting and add new proxies after i hit stop/pause harvesting it freezes again.. So is there any way how to set GSA SER to use more threads to scraping urls?

mindmaster · Sep 12, 2015

I understand your scrapebox is freezing?
What are your pc/vps specs? Are you using the latest version of scrapebox?

zyx2000 · Sep 12, 2015

Could it be low memory?

loopline · Sep 13, 2015

If you don't have enough resources Scrapebox can freeze. I can't help you get GSA faster, but your Scrapebox problem is probably solveable.

If you get 64bit, assuming you have a 64bit OS, then it can use as much resources as you have, so that could help. If you don't have enough resources then you will have to just work with smaller lists.

You could make small automator runs to do scraping and then when its done just swap the keywords in all the files and repeat.

Also you don't need to pause to add more proxies, you can use the auto load proxy from file option in the harvester and Scrapebox will automatically reload proxies from file when it needs it. Or if you are using the detailed harvester you can set it to reload every X minutes.

so using options like these should solve the freezing issue, but its likely due to lack of system resources anyway.

fullwall3t · Sep 13, 2015

Thanks for trying to help, my mistake i ahould wrote down specs if my vps.. Hw is not an issue, 8gb ddr3 1800mhz, quad core 3.5ghz, internet is about 20mb/s. i tried to lower number of threads in scrapebox.. it doesnt help, still same issue, with maximum threads i don't exceed usuall more than 50% of hw usage

loopline said:
If you don't have enough resources Scrapebox can freeze. I can't help you get GSA faster, but your Scrapebox problem is probably solveable.

If you get 64bit, assuming you have a 64bit OS, then it can use as much resources as you have, so that could help. If you don't have enough resources then you will have to just work with smaller lists.

You could make small automator runs to do scraping and then when its done just swap the keywords in all the files and repeat.

Also you don't need to pause to add more proxies, you can use the auto load proxy from file option in the harvester and Scrapebox will automatically reload proxies from file when it needs it. Or if you are using the detailed harvester you can set it to reload every X minutes.

so using options like these should solve the freezing issue, but its likely due to lack of system resources anyway.

loopline · Sep 14, 2015

does it popup a error window? If so there is a bug report file called bugreport.txt, its in your main Scrapebox folder >> configuration folder.

Send that to scrapeboxhelp (at] gmail {dot) com

If there is no error box, can you post a screenshot or a link to one?

Faster url/list scraping? PLS HELP

fullwall3t

BANNED

mindmaster

Elite Member

zyx2000

Registered Member

loopline

Elite Member

fullwall3t

BANNED

loopline

Elite Member

Main Menu

Marketplace

Making Money

BlackHat World