VPS for ScrapeBox - A Stable One

I need a VPS that can hold on to thousands of keywords (maybe close to million) and run for months.

Is there a VPS that's good enough for this?

The one we are currently using just breaks all the time, ScrapeBox is always "Not Responding" and the PC is restarting.

Any suggestions?

Nikola

memory is going to be your first hurdle, so millions of keywords, Id start with 8GB of memory and see if that works for you.

Also you would be better to run multiple instances and divide your connections and keywords among them. Windows "likes" this better and if 1 crashes you still have others running.

i join the question

loopline said:

memory is going to be your first hurdle, so millions of keywords, Id start with 8GB of memory and see if that works for you.

Also you would be better to run multiple instances and divide your connections and keywords among them. Windows "likes" this better and if 1 crashes you still have others running.

Click to expand...

Yes but this will require from us to have more proxies?

I'm thinking of adding 800k keywords - that's the amount of keywords we are using right now to Scrape the needed data.

We just got the automator. I'm thinking 10 separate automator jobs with 800k keywords each. Will this work?

We can try with 8GB.

Is, like, automator a separate instances running? They are consecutive. Will this work?

Is there a VPS provider that you would recommend to us?

Thanks

NikolaKica said:

Yes but this will require from us to have more proxies?

I'm thinking of adding 800k keywords - that's the amount of keywords we are using right now to Scrape the needed data.

We just got the automator. I'm thinking 10 separate automator jobs with 800k keywords each. Will this work?

We can try with 8GB.

Is, like, automator a separate instances running? They are consecutive. Will this work?

Is there a VPS provider that you would recommend to us?

Thanks

Click to expand...

Proxies are generally related to connections as most providers have a max simultaneous connections limit. So if you have 10 proxies with each one having a max connections limit of 10, then that gives you 100 total connections and if your running 10 instances you can set each instance to 10 connections and put all the proxies in each instance, assuming they are private. But you won't be able to run 10 connections on 10 proxies scraping google anyway.

Your putting all 800 keywords in each instance? or 800K different keywords in each instance?

Each instance can run its own automator job. The great thing about a vps is you can generally scale up resources pretty each. Here is some good providers
http://scrapeboxfaq.com/scrapebox-vps-hosting
Im not sure I follow though your automator setup, what do you mean by consecutive? You want to run 10 instances all at the same time or only 1 instance at a time and run thru 10 of them in order 1 by 1?

loopline said:

Proxies are generally related to connections as most providers have a max simultaneous connections limit. So if you have 10 proxies with each one having a max connections limit of 10, then that gives you 100 total connections and if your running 10 instances you can set each instance to 10 connections and put all the proxies in each instance, assuming they are private. But you won't be able to run 10 connections on 10 proxies scraping google anyway.

Your putting all 800 keywords in each instance? or 800K different keywords in each instance?

Each instance can run its own automator job. The great thing about a vps is you can generally scale up resources pretty each. Here is some good providers
http://scrapeboxfaq.com/scrapebox-vps-hosting
Im not sure I follow though your automator setup, what do you mean by consecutive? You want to run 10 instances all at the same time or only 1 instance at a time and run thru 10 of them in order 1 by 1?

Click to expand...

You know loopline I can manage with this.

I found a good VPS and I guess it doesn't matter actually how many instances we run I think only a stable VPS will do the trick.

But you could help me with another question if you could spare time.

I put a keyword like: car
I add all USA cities: car new york, car las vegas
Then I add 10 stop words like: car new york 1, car new york 2, car new york about

And I scrape with those, approximately 100k, or 200k keywords or sometimes 400k or 500k

but I don't get too much results.

I'd like to get around 50k contact forms every 3 days for an example.

What would be the correct steps to get 50k contact forms in 2-3 days?

Can you help me with this?

Maybe scrape more keywords on the keyword of CAR then add cities? Then add stop words?

Maybe get 200 keywords that are related to CAR and then only add cities to them?

Any suggestions?

Thanks

@NikolaKica you need footprints for contact forms. In footprint example above you dont have anything contact form related. You can also scrape CMS where contact forms are quite common and look for contact form on given site with different software.

satyr85 said:

@NikolaKica you need footprints for contact forms. In footprint example above you dont have anything contact form related. You can also scrape CMS where contact forms are quite common and look for contact form on given site with different software.

Click to expand...

Right now I've been scraping only websites and plan to put them in the GSA contact form submitter after the scrape. But Usually I've been just selecting the Platforms tick on Scrapebox and I've selected the Wordpress Contact Forms + I've added to the Wordpress file extra footprints I found on the looplines website.

Any suggestions?

Go with Hetzner and find a good server from the server auction. I got one with 32 GB RAM and never had any problem running multiple instances of SB on it.

V said:

Go with Hetzner and find a good server from the server auction. I got one with 32 GB RAM and never had any problem running multiple instances of SB on it.

Click to expand...

But do we have to pay extra for the OS ?
or can we install our own OS ?

miyuru2u said:

But do we have to pay extra for the OS ?
or can we install our own OS ?

Click to expand...

It's a little bit difficult to install your OS but you can do it. If you take a genuine copy of Windows from them, then you have to pay extra.

Most of the core process for the Scrapebox Harvester consumes CPU, so I'm not sure why Loopline is talking about RAM, but he's certainly a pro.
Some of the Addons shift into RAM a lot more than CPU, so maybe that part of Scrapebox uses RAM more than CPU... not sure... but I guess you'll find-out.

What I would say, though, is your problem isn't necessarily the server itself, but rather the biggest issue Scrapebox has, in general, which is threadlocks.
According to support, that actually is a problem with how Scrapebox interacts with the Windows API.
There are many thousands of threads available from Windows, but a single instance of Scrapebox taps into somewhere between 200 to 500 connections on average, depending on the feature you are using.
So, it's probably like LL says, where you will wanna run a bunch of instances, and see what you can manage to pull at-scale, since some of those will inevitably freeze from threadlocks.

As far as your machine goes, I no longer use VPS services, but rather purchased refurbished machines from Ebay.
I like the HP Elitedesk 800 g1, which ideally comes with an i5 4th gen CPU, and 16gb RAM.
You can usually get one for about $300, give or take.
Here's my video on the topic:

I don't know how many instances you could do with your process.
Once you exceed 80% CPU usage, your chances of system crash and reboot are quite high.
So, running multiple instances might not be practical, but it totally depends on how many threads you are running at the same time, and how system intensive that part of SB might be on system resources.

What you need to do is run a single instance test, and make note of how much system resources are utilized.
You might also consider using the premium Automator plugin to try and run a small number of instances in sequence.
However, if any part of your sequence addons fall victim to threadlocks, then the whole sequence will basically freeze.

Depending on your budget, you might consider getting a few machines, and running a couple different sequences at a time.
I would question whether or not all that is even necessary, though. (LoL)

Why don't you instead get a dedicated server instead of VPS? I've used OVH before without problems but you can shop around for prices. Just search for reviews from members here on the forum

Contabo VPS, effective and low cost.

NikolaKica said:

You know loopline I can manage with this.

I found a good VPS and I guess it doesn't matter actually how many instances we run I think only a stable VPS will do the trick.

But you could help me with another question if you could spare time.

I put a keyword like: car
I add all USA cities: car new york, car las vegas
Then I add 10 stop words like: car new york 1, car new york 2, car new york about

And I scrape with those, approximately 100k, or 200k keywords or sometimes 400k or 500k

but I don't get too much results.

I'd like to get around 50k contact forms every 3 days for an example.

What would be the correct steps to get 50k contact forms in 2-3 days?

Can you help me with this?

Maybe scrape more keywords on the keyword of CAR then add cities? Then add stop words?

Maybe get 200 keywords that are related to CAR and then only add cities to them?

Any suggestions?

Thanks

Click to expand...

How many contact forms related to cars are there?

Thing is there is probably only 30 million contact forms world wide on top level domains. So think about how few related to cars there are.

For example I did
car inurl:contact

in google and then click thru to the last page. Ignore what it says for the count at first its X millions but when you click thru it cuts you off at page 4 and says
Page 4 of about 348 results (0.83 seconds)

thats not a lot of results. Surely there are more then 348 contact forms related to cars, and your doing good with stop words, but Im not sure you will find 50K contact forms every 2 -3 days related to cars, at least not for a long time, your going to quickly be into duplicates.

But I think the way you are doing it is somewhat ok, the key is footprints, the better the footprint the more accurate result your going to get.

Unless you just do mass, then of course you can go that route too.

If you want to try your hand at footprints here is a video


@getivan
love the pig picture, lol

loopline said:

How many contact forms related to cars are there?

Thing is there is probably only 30 million contact forms world wide on top level domains. So think about how few related to cars there are.

For example I did
car inurl:contact

in google and then click thru to the last page. Ignore what it says for the count at first its X millions but when you click thru it cuts you off at page 4 and says
Page 4 of about 348 results (0.83 seconds)

thats not a lot of results. Surely there are more then 348 contact forms related to cars, and your doing good with stop words, but Im not sure you will find 50K contact forms every 2 -3 days related to cars, at least not for a long time, your going to quickly be into duplicates.

But I think the way you are doing it is somewhat ok, the key is footprints, the better the footprint the more accurate result your going to get.

Unless you just do mass, then of course you can go that route too.

If you want to try your hand at footprints here is a video

@getivan
love the pig picture, lol

Click to expand...

Thanks loopline

miyuru2u said:

But do we have to pay extra for the OS ?
or can we install our own OS ?

Click to expand...

It is not so difficult to install your own copy of Windows server on hetzner. Copies of Windows can run for 6 months, after that you can expect problems without a key. Keys you can buy from Ebay.

I buy all my windows keys from ebay for $15 or less. Never had a single issue. Just follow directions and activate it in 48 hours or less. I run the latest versions of windows server 2019.

My host just sets it up for me unactivated and then I buy a key, activate and saves me the standard $15+ per month that most places charge for windows, first month and its more then paid for.