Hi loopline,
Maybe you have encountered such situation. I imported 200,000 links to GSA website contact, but after a few hours, only 180,000 were left in the software. I don't know where the 20,000 links went. How did you solve it?
Scrapebox crashes every three days when I scrape for links. . My computer: intel core i5-3230 2.6Gz, RAM 6.00GB.
The bugreport is here https://drive.google.com/file/d/1cZBlgCWNkdxTMk1rwGZ7l7nvE2_1eElm/view?usp=sharing
And every time I adjusted the scrape thread to 100 +, the scrapebox CPU usage was 80% and I had to run three scrapeboxs at once. Processing data is a hassle.
In addition, how should I custom the test to get proxy for scraping keywords? I now have to use a VPN to scrape for keywords
Thanks for your time,
John
Never had that happen. Perhaps they are still there but they just got sorted into another column, like failed or filtered etc...
You only have 6GB of memory if your leaving it run for days its probably running out of resources.
Edit: just read your bug report, it shows exactly that, out of resources. You have 35GB of free disk space, which honestly isn't all that much, but if your just scraping is probalby enough, but your likely running out of memory (RAM). So you want to work with smaller lists of keywords.
Also your running Scrapbox from your D drive, which is not ideal, you want to run it from the same drive as windows or windows will cause issues. Windows doesn't like EXE files running from other drives, ideally. So I would move your scrapebox folder to a folder on your desktop or in your documents folder. You can still export and import files from the D drive though.
I mean your running an i5 with a benchmark of 3939, and only 6Gb of memory. So your basically using a plain old hammer and nails to try and build a house and getting frusterated because you expect the house to go up as fast and smoothly as if your using a pneumatic framing nailer and an assistant to help. Ive been there and done that. These days I have the equivalent of a whole crew of construction people helping me put up the house with all the best tools and I have the bills for it all to boot, but I get the houses up faster too and thus more money. If you follow my example, hopefully it makes sense. But you have to work up to that point.
In short, your machine is fairly small and low on physical resource specs for what your talking about trying to do. So yeah, its going to be frustrating to deal with data. Your only choices are work with smaller lists, be frusterated or slap down some dollars for a "bigger", faster, better computer to work on. That or rent a VPS/Server
You can't build a custom test for a VPN, only for proxies. So for a VPN, your still only using 1 ip, its just like using no VPN or Proxies functionally, but of course if you burn an ip then you can easily switch to a new one.
Scraping keywords are fairly basic and lenient on ip bans. So just scrape, if it works great, if it doesn't, swap ips with your vpn provider and try again. I assume your VPN lets you swap ips out via their software in like 60 seconds or so.
So as in all things life and business there are 2 currencies, money and time.
You have to spend more of 1 or the other, or have a nice balance. If you don't have the money to buy the great tools or rent them (servers, computers, proxies etc..) then you have to spend the time. The time changing out ips manually via the VPN, waiting impatiently while data processes etc...
Given enough hard work the obvious goal is to get enough money to upgrade to the tools that free up your time but still make enough money. Ive done the grunt work, much more manual then you are doing now, its not fun, but you do what you must until you can do it different or better.
There are no shortcuts.
