How to get 8,000,000 keywords from SemRush?

[arganrecords]

As you all know, SemRush allow to download a report (or view it) of a maximum 30K keywords (Guru Plan) but I need to get all Yahoo's keywords ranked in the Google search.
Now, there is a powerful tool or it's impossible to get this amounth of keywords?

Thanks for all the suggestions.

 

[leo29]

This may not be a perfect suggestion for your requirement but I think we could search for yahoo com in semrush for different countries (provided in semrush options) and get more list than 30k. However this may not come near 7 digit number but could do much more than 30k

 

[nanavlad]

Sorry I can't help you.
But I.m intrigued, What the hell are you going to do with 8 million keywords

 

[arganrecords]

This may not be a perfect suggestion for your requirement but I think we could search for yahoo com in semrush for different countries (provided in semrush options) and get more list than 30k. However this may not come near 7 digit number but could do much more than 30k

I tried to take more possible keywords setting the first and last otherwise placing the levels of the data. So in this way I get 180K keywords but it isn't enought.

Sorry I can't help you.
But I'm intrigued, What the hell are you going to do with 8 million keywords

I need to do keyword research on Yahoo.com to discover the most visited pages in my niche.

 

[RxAff]

Forget about Yahoo, if you meant yahoo answers ))) They will ghost all your answers and will ban your account. If you want traffic to your website just fresh answers will give it to you.

 

[FBGuru]

Argan, The tools of the trade lies within their own API.

Let me explain how you do it.

You should use the domain_organic request in their API to pull the data.

Here's an example :

http://api.semrush.com/?type=domain_organic&key=INSERTYOURAPIKEYHERE&display_filter=%2B%7CPh%7CCo%7Cseo&display_limit=10&export_columns=Ph,Po,Pp,Pd,Nq,Cp,Ur,Tr,Tc,Co,Nr,Td&domain=seobook.com&display_sort=tr_desc&database=us

^ This request will pull the following data.

As you can see, it pulled out data for the first 10 keywords. (Ignore the results though as I used seobook.com in the example) You can increase the total number of keywords exported using the display_limit parameter. In the above example, I used 10(see: display_limit=10). You can try it with 100000. I don't know the maximum number but you can talk with their support and see how much data they allow to export per request.

Say, you pulled the first 100000 keywords using

http://api.semrush.com/?type=domain_organic&key=INSERTYOURAPIKEYHERE&display_filter=%2B%7CPh%7CCo%7Cseo&display_limit=100000&export_columns=Ph,Po,Pp,Pd,Nq,Cp,Ur,Tr,Tc,Co,Nr,Td&domain=yahoo.com&display_sort=tr_desc&database=us

What you've to do in order to pull the NEXT 100000 results is by simply adding another parameter called display_offset into your request. Here's an example :

http://api.semrush.com/?type=domain_organic&key=INSERTYOURAPIKEYHERE&display_filter=%2B%7CPh%7CCo%7Cseo&display_limit=100000&display_offset=100000&export_columns=Ph,Po,Pp,Pd,Nq,Cp,Ur,Tr,Tc,Co,Nr,Td&domain=yahoo.com&display_sort=tr_desc&database=us

As you can see, I added display_offset=100000. What it basically does is, it skips the first 100000 results and pulls the next 100000 results.

So now you have results for the first 200,000 keywords. Repeat the requests by simply increasing the display_offset to get more keywords.

This is how your request should look like :

First request :

http://api.semrush.com/?type=domain_organic&key=INSERTYOURAPIKEYHERE&display_filter=%2B%7CPh%7CCo%7Cseo&display_limit=100000&export_columns=Ph,Po,Pp,Pd,Nq,Cp,Ur,Tr,Tc,Co,Nr,Td&domain=yahoo.com&display_sort=tr_desc&database=us

Second request :

http://api.semrush.com/?type=domain_organic&key=INSERTYOURAPIKEYHERE&display_filter=%2B%7CPh%7CCo%7Cseo&display_limit=100000&[B]display_offset=100000[/B]&export_columns=Ph,Po,Pp,Pd,Nq,Cp,Ur,Tr,Tc,Co,Nr,Td&domain=yahoo.com&display_sort=tr_desc&database=us

Third request :

http://api.semrush.com/?type=domain_organic&key=INSERTYOURAPIKEYHERE&display_filter=%2B%7CPh%7CCo%7Cseo&display_limit=100000&[B]display_offset=200000[/B]&export_columns=Ph,Po,Pp,Pd,Nq,Cp,Ur,Tr,Tc,Co,Nr,Td&domain=yahoo.com&display_sort=tr_desc&database=us

Fourth request :

http://api.semrush.com/?type=domain_organic&key=INSERTYOURAPIKEYHERE&display_filter=%2B%7CPh%7CCo%7Cseo&display_limit=100000&[B]display_offset=300000[/B]&export_columns=Ph,Po,Pp,Pd,Nq,Cp,Ur,Tr,Tc,Co,Nr,Td&domain=yahoo.com&display_sort=tr_desc&database=us

It costs you 10 API units per line so in order to pull 100000 keywords, you need 1 Million API Units but the Guru plan comes with just 500,000 which is good enough only to pull 50k keywords.

They have separate pricing for their API though.

http://www.semrush.com/api_products.html

It shows 8,081,712 organic keywords in their US database for Yahoo.com. So in order to export all the keywords, you'd need 80 Million+ API Units. Their biggest API Package comes up with 100 Million API Units which will set you back by $3500.

Hope this helps.

 

[webdirect]

That many keywords sounds extreme, it will be interesting to hear what you plan to do with that many keywords. Are you going to sift through them to find which ones are the easiest to compete for?