Scrapebox Expired Domain Finder

[Kanye West]

No, there is no setting that would cause that. That makes no sense.

So when you run it, manually, you mean you choose to load domains and recehck metrics and it works then but not when your just running along scraping for the expired domains in the first place?

It works only if I click on the button "Majestic metrics" but not while scraping (all fields are 0).

Btw ram usage of the expired domain finder is getting higher and higher, will this stop? This scraping run is active since ~26 hours but there is one month to go likely and ram will be full if this continues over the next 24 hours.

Edit: Ram usage seems to stabilize.

 

[zencontent]

I have to agree it is a fantastic product - one little addition that would be good (I've done my own code for this but nice to see it in SB) would be to grab the title from the last archive.org entry. It would give you an idea if the site has been pillaged before, and cutting and pasting across to archive is just a drag.

 

[benxh]

I use it daily and scrap for thousands of expired domains at a time.
In this last scan it ran for one day and found 4437 expired domains:
View attachment 81221

It does find ever domain extension available, ones you can register and ones you cannot.
After I filter down to what I want ( .com, .net, .org, .info, .biz, .us even) the list becomes 3254 expired domains:
View attachment 81222

Exporting options are great and easy to use with all metrics listed. Even the Archive.org WayBack machine snapshots
View attachment 81223

The one thing I would like to find is a PHP script that lets me import this list into a website to list the expired domains and metric, with the domain names hidden, sell the expired domains.

I love ScrapBox and highly recommend the tool and this Premium Plugin

Psykeus

I don't know know if you found a script, but I could probably whip a very basic one up for you (idk if I can PM yet, just send me a basic list (hide the domains) of say 10 entries, and I'll make a script that will import a csv, or whatever you have, into a database, and a simple PHP script that will list the domains by other information they have, without showing the domain, but by offering a PP or whatever buy now link for that specific domain.

 

[loopline]

It works only if I click on the button "Majestic metrics" but not while scraping (all fields are 0).

Btw ram usage of the expired domain finder is getting higher and higher, will this stop? This scraping run is active since ~26 hours but there is one month to go likely and ram will be full if this continues over the next 24 hours.

Edit: Ram usage seems to stabilize.

You can go into your scrapebox folder >> plugins >> expired finder folder and delete the majesticcookies.txt file

Scrapebox stores a cookie for majestic because majestic limits logins to avoid account sharing etc.. So If scrapebox logs in to much it will get the account locked so hence the cookie. But you can shut down the plugin, delete the cookie file and then try it, because if that cookie file gets messed up then it will cause issues.

Doesn't really make sense why it works manually but not when scraping, but its the only thing I know to try.

 

[Kanye West]

You can go into your scrapebox folder >> plugins >> expired finder folder and delete the majesticcookies.txt file

Scrapebox stores a cookie for majestic because majestic limits logins to avoid account sharing etc.. So If scrapebox logs in to much it will get the account locked so hence the cookie. But you can shut down the plugin, delete the cookie file and then try it, because if that cookie file gets messed up then it will cause issues.

Doesn't really make sense why it works manually but not when scraping, but its the only thing I know to try.

Thanks for this tip. I will try it but I'm skeptical. It only works sometimes if I click on "Majestic metric" but even then not always.

What I noticed is that if I have two instances open on the same server the second instance will eat all of my ram. First instance is scraping at full speed and uses max 1GB and second instance is scraping much slower but ram usage is at 10GB. I have to stop it else it would use the whole 16GB RAM.

 

[loopline]

Thanks for this tip. I will try it but I'm skeptical. It only works sometimes if I click on "Majestic metric" but even then not always.

What I noticed is that if I have two instances open on the same server the second instance will eat all of my ram. First instance is scraping at full speed and uses max 1GB and second instance is scraping much slower but ram usage is at 10GB. I have to stop it else it would use the whole 16GB RAM.

Try a test, run 2 instance, and load the exact same starting url and same identical settings for both instances. Does it still happen?

Because if your running different source urls then your parsing different pages and the memory usage would be different so its hard to really sort what may be wrong

 

[jon_xx_x]

This plugin is PURE GOLD man! First run and I get my hands on TF 28 DA 40 expired domain name. I can easily sell it for 50 USD above. This plugin paid for itself in an hour.

Guarantee that domain is shit. Guaranteed.

 

[nobodyelsein]

I'm a bit late to the party with this plugin. Scrapebox has become slightly less useful to me since Moz tightened up their API and the backlink harvester plugin stopped working.

This plugin could easily make Scrapebox essential to me again.

 

[loopline]

I'm a bit late to the party with this plugin. Scrapebox has become slightly less useful to me since Moz tightened up their API and the backlink harvester plugin stopped working.

This plugin could easily make Scrapebox essential to me again.

The plugin didn't stop working, it works fine and has been working fine for months actually.

Make sure you are using the latest version, as old versions don't work.

If so what errors are you getting? Screenshots welcome.

 

[ni_luan]

Guarantee that domain is shit. Guaranteed.

How can you tell ? Please enlighten me.

 

[nobodyelsein]

The plugin didn't stop working, it works fine and has been working fine for months actually.

Make sure you are using the latest version, as old versions don't work.

If so what errors are you getting? Screenshots welcome.

My apologies, it is working again. Which is brilliant news!

Thanks (as always).

 

[loopline]

My apologies, it is working again. Which is brilliant news!

Thanks (as always).

Great glad its working for you.

 

[nobodyelsein]

I have just paid for the expired domain plugin. When my licence is activated I'll let you know how I get on.

I just found an old money site I'd forgotten about. It had just one visit this month! I'm gong to point a couple of relevant expired domains at it and see what happens. This is clearly not the only reason I bought he plugin but I thought it would be a fun experiment.

 

[loopline]

I have just paid for the expired domain plugin. When my licence is activated I'll let you know how I get on.

I just found an old money site I'd forgotten about. It had just one visit this month! I'm gong to point a couple of relevant expired domains at it and see what happens. This is clearly not the only reason I bought he plugin but I thought it would be a fun experiment.

sounds good. Indeed a good experiment, experiments are the best because you often learn things you didn't know you didnt know.

 

[wendoraaf]

How much is the plugin?

 

[loopline]

How much is the plugin?

$47 or there is a bundle you can get thats close to half off that includes the expired finder

http://www.scrapebox.com/plugins

 

[nobodyelsein]

I just gave the expired domain finder it's first twirl. I entered six domains that are authoritative in my niche and it found zero expired domains. Then I realised I had to set up the plugin properly and add MOZ api keys (sadly I don't have a Majestic account). I then set the scan depth to '10' and pressed 'start'. Over 24 hours later the plugin was still finding expired domains! I pressed 'stop' when 470 domains were found and had a look at the metrics.

I have registered a couple of domains that are 'okay' which I'm happy with for my first attempt. DA is only around 30 but the domain that is hosting the link is very on point for my niche, so I've gone for relevance rather than brute force.

I'm going to run the scraper again on just one domain and a depth of 5 to see what happens. I'll report back.

 

[mtj85]

HI. I've purchased this plugin.

I have a problem with it. When I select 1 level deep crawling the plugin would multi-thread (50) and works pretty fast.
But when i select 2 or more levels deep the plugin would not multi-thead (it always shows only 1 thread running).

Have I configured it wrong or is this work as intended?
I know this is not a support thead, but this reply might help some other users as well.

Thank you.

 

[loopline]

I just gave the expired domain finder it's first twirl. I entered six domains that are authoritative in my niche and it found zero expired domains. Then I realised I had to set up the plugin properly and add MOZ api keys (sadly I don't have a Majestic account). I then set the scan depth to '10' and pressed 'start'. Over 24 hours later the plugin was still finding expired domains! I pressed 'stop' when 470 domains were found and had a look at the metrics.

I have registered a couple of domains that are 'okay' which I'm happy with for my first attempt. DA is only around 30 but the domain that is hosting the link is very on point for my niche, so I've gone for relevance rather than brute force.

I'm going to run the scraper again on just one domain and a depth of 5 to see what happens. I'll report back.

Good deal, glad its making progress. It takes some trial and error to get it dialed into what will work best for your specific needs. But it sounds like your well on your way.

HI. I've purchased this plugin.

I have a problem with it. When I select 1 level deep crawling the plugin would multi-thread (50) and works pretty fast.
But when i select 2 or more levels deep the plugin would not multi-thead (it always shows only 1 thread running).

Have I configured it wrong or is this work as intended?
I know this is not a support thead, but this reply might help some other users as well.

Thank you.

If your using only level 1 its only looking at the urls that you have loaded, so it can go really fast. If you select multiple levels then its going to use 1 thread to load 1 url, then extract internal and external links. Then say it has 30 links to process it would use 30 threads, and then go from there until it has enough threads to reach the max you have set. As you let it go you will see the thread count going up and down some as it works as effeciently as possible.

IF it always only stays on 1 thread then its possible something is wrong, like the connections are being blocked and so it can't ever find any internal links. So make sure you whitelist the plugin in all security software.

 

[mtj85]

HI. Thanks for replying. Yes, I understand the concept of deep-crawling.

But the problem is that when I load lets say 50 URL as seed urls and set 2 levels deep, I would expect that 50 threads are created and deep crawling the 50 seed links separately.

But like I said, if I set level2 crawling on 50 URL it only uses 1 thread and crawls the URLs in cascade mode. It does not use several threads to crawl multiple seed urls at the same time. But If I use 1 level deep crawling the 50 URLs are crawled quickly by 50 threads at one time like they should.

So what seems to be the issue here?

Thank you for your response.