Overcoming new MOZ API limitations

[agentk007]

So I've started to get hard core into scrapebox - especially for finding expired web 2.0 domains that have strong metrics. Been having GREAT success, except now where I have about 10 million unique web 2.0 domains to sort through. I know I'll first sort them by availability using the vanity checker, but what is your next step?

I've always run them through MOZ next but with the new limitations on their free APIs, I find it hard to process large quantities of data - even with the handful accounts I already have. Of course paying is an option but their fees are a little far out there in MHO.

How do/would you sort through something like this?

 

[loopline]

Thats a hard question. Moz is the best free option and as you noted they now have way lower limitations. Going from 1 million requests per month to only 25K they are clearly trying to limit free requests.

So paying is an option but there is no great free mass option out there. Page rank is gone now so you can't even use that. I know Scrapebox is looking into options out there and more may come with time, but right now its in a state of flux so I would say do the best you can.

But if you sort it down to whats available with the vanity checker you probably wont' have all that many tens of thousands and you can do 25K on 1 moz account so if you have a few then your probably good for now Id guess?

 

[agentk007]

Thats a hard question. Moz is the best free option and as you noted they now have way lower limitations. Going from 1 million requests per month to only 25K they are clearly trying to limit free requests.

So paying is an option but there is no great free mass option out there. Page rank is gone now so you can't even use that. I know Scrapebox is looking into options out there and more may come with time, but right now its in a state of flux so I would say do the best you can.

But if you sort it down to whats available with the vanity checker you probably wont' have all that many tens of thousands and you can do 25K on 1 moz account so if you have a few then your probably good for now Id guess?

Great hearing from you again Loopline. Also great to hear ScrapeBox is looking at alternatives.
And you're right: Once cut down to whats available using the vanity checker, it IS still manageable. I was somehow under the impression they only allowed 10,000 requests now, but you're right: it's 25k.

I remember maxing out my accounts when I sorted blogs for manual commenting. So I tried running them through Alexa first just to get the URL volume down, but I doubt Alexa would be of much use for sorting expired web 2.0s.

 

[The Curator]

Maybe do a group buy? I see them prevalent on another forum...

 

[agentk007]

Maybe do a group buy? I see them prevalent on another forum...

Interesting thought. Almost doesn't look like it would be worth it though... looking at $500 for 500,000 rows

 

[agentk007]

Speaking of free moz account: I've been trying to run a scan for a couple days now... and eventhough these accounts have lots of room for it, all it tells me for each URL is: 0

No PA, no DA, nothing. Tried different APIs and still same thing. Any idea why that might be?
Thought it might be the IP address I'm running it from so I tried proxies... and still get the same results. Super confused!

 

[Ambitious12]

Find and make a list of that domains then start checking on opensiteexplorer then you would know about the perfect domain on that all.

 

[yasburrows]

Interesting thought. Almost doesn't look like it would be worth it though... looking at $500 for 500,000 rows

Where are you getting those figures from? I did a group buy through www.seogroupbuy.com (not affiliated in ANY way) and they have a majestic gold, moz medium and semrush pro for $17 pm

 

[The Owl]

You could allways just pay for it problem solved sometimes you have to spend money!

 

[agentk007]

Where are you getting those figures from? I did a group buy through www.seogroupbuy.com (not affiliated in ANY way) and they have a majestic gold, moz medium and semrush pro for $17 pm

Too funny you should mention them. We're working on a JV of sorts (between my site and theirs). That reminds me... they may be waiting on an email response from me. lol

$17 / month for Majestic is peanuts. Worth it. I honestly wish scrapebox had a majestic addon!

 

[agentk007]

You could allways just pay for it problem solved sometimes you have to spend money!

Would it solve the API issue I'm having?

 

[The Owl]

Would it solve the API issue I'm having?

Well you said the problems with there free api is there a paid version?

 

[The Curator]

You could allways just pay for it problem solved sometimes you have to spend money!

Nice addition to the thread captain obvious..

 

[agentk007]

Well you said the problems with there free api is there a paid version?

Well I am having issues with the new Moz limits... but the other issue I'm currently having is that scrapebox won't do a proper scan. I left a comment on the scrapebox thread, but not sure the issue is with them, with Moz, or even with me.

Here's a screenshot of what I mean:

It keeps giving me 0s... that's all. When I manually paste the URLs in Moz search bar, they have decent metrics. Can't figure out why it does that. It only started doing this about 3 days ago. Tried a new account, tried different IP addresses... tried different scrapebox instances... no luck!

 

[loopline]

Speaking of free moz account: I've been trying to run a scan for a couple days now... and eventhough these accounts have lots of room for it, all it tells me for each URL is: 0

No PA, no DA, nothing. Tried different APIs and still same thing. Any idea why that might be?
Thought it might be the IP address I'm running it from so I tried proxies... and still get the same results. Super confused!

Moz installed incapsula just before they reduced the limit to 25K. For a good bit there they had it turned up an incapsula was blocking lot of valid requests from all sorts of tools, including scrapebox, in a browser, etc...

Well lots of Sbox users and other tool users around the globe emailed them about it and after a bit they backed off and things started working.

Well now they must have it turned back up because again they are blocking valid requests even in a browser, and of course in scrapebox etc...

The best I could say is to mail them and ask if your moz account/ip can be whitelisted to not get blocked. But they are on a mission to eliminate as much free traffic as they can it seems.

Good to see your still around as well mate.

 

[rpkrocks]

After you find the available Web 2.0s, use URL Profiler to check them through Moz. It has the capability of holding a 5 seconds gap between each usage of 10 API credits.

 

[agentk007]

Moz installed incapsula just before they reduced the limit to 25K. For a good bit there they had it turned up an incapsula was blocking lot of valid requests from all sorts of tools, including scrapebox, in a browser, etc...

Well lots of Sbox users and other tool users around the globe emailed them about it and after a bit they backed off and things started working.

Well now they must have it turned back up because again they are blocking valid requests even in a browser, and of course in scrapebox etc...

The best I could say is to mail them and ask if your moz account/ip can be whitelisted to not get blocked. But they are on a mission to eliminate as much free traffic as they can it seems.

Good to see your still around as well mate.

Bammm.... you have little birdies everywhere, don't you Loopline?! lol (a Game of Thrones reference).

That explains a lot. I suspected something like that was happening... I just might be forced to do what they want me to do and pay for an upgraded account. Seeing how I could process a lot more data a lot quicker... this MAY be worth it.

Thanks again!

 

[Ambitious12]

If you want everything at once then i too recommend MOZ is one of the best platform where you can find everything.

 

[agentk007]

Just an update: I submitted a ticket to Moz and asked them to whitelist my IP. I ended up getting an upgraded account via a groupbuy (thanks for the recommendation yasburrows) and that too returned the same results, so it is likely my VPS IP that got flagged.
The group buy looks solid though... would definitely recommend it from what I have seen so far!

 

[rpkrocks]

Just an update: I submitted a ticket to Moz and asked them to whitelist my IP. I ended up getting an upgraded account via a groupbuy (thanks for the recommendation yasburrows) and that too returned the same results, so it is likely my VPS IP that got flagged.
The group buy looks solid though... would definitely recommend it from what I have seen so far!

Hey agent, I have already provided you the solution in previous post. why are you not using that?