competitor ranking keywords ?

seo-limited · Jun 3, 2015

hello , hmm I see now so they are just using other services not making some kind of bot or crawler to gather the data themself (sourcehunter)

so now I want to know how do semrush do to get competitor data ? I want to make my own script to gather the data myself is it possible ?

I don't want to use semrush/ahrefs services

Wilson Grant Fisk · Jun 3, 2015

seo-limited said:
...
I want to make my own script to gather the data myself is it possible ?
...

Yes, it is possible, but sorry to be blunt, if you had the skills to do it, you wouldn't be asking the question.
Unless you are willing to invest considerable time and money to this, it won't be worthwhile, it will be easier and cheaper to use the API's.

rank bank · Jun 3, 2015

Most probably you can check through "SEMRUSH"

netmoney1 · Jun 4, 2015

seo-limited said:
hello , hmm I see now so they are just using other services not making some kind of bot or crawler to gather the data themself (sourcehunter)

so now I want to know how do semrush do to get competitor data ? I want to make my own script to gather the data myself is it possible ?

I don't want to use semrush/ahrefs services

lol this gets more comical by the post. Hire an engineer and sink $50k and make your own magical metric scraper.

sinking50s · Jun 4, 2015

Semrush is the only tool that does what you are looking for, anything else is just an api tool.

seo-limited · Jun 4, 2015

netmoney1 said:
lol this gets more comical by the post. Hire an engineer and sink $50k and make your own magical metric scraper.

Hello , I'm an engineer myself I don't need to hire one , I'm still thinking about this project , because ahref/majestic/semrush bots are known among the black hat community which make them useless to detect backlinks - many pbn owners use htaccess to deny those bots from crawling -

KHer0 · Jun 5, 2015

seo-limited said:
Hello , I'm an engineer myself I don't need to hire one , I'm still thinking about this project , because ahref/majestic/semrush bots are known among the black hat community which make them useless to detect backlinks - many pbn owners use htaccess to deny those bots from crawling -

No one knows your ability and resources but I really think it's not possible for you to do it considering the questions you are asking.

They have crawlers just like google, and they keep checking top 20 ranking keywords in google and save the data. When you enter a keyword, they retrieve the urls shown for this url. The efficiency of this depends on the efficiency of the crawler. Right now they have +10 billion keywords indexed. Can you create a crawler and database to handle this volume ?!

Wilson Grant Fisk · Jun 5, 2015

seo-limited said:
Hello , I'm an engineer myself I don't need to hire one , I'm still thinking about this project , because ahref/majestic/semrush bots are known among the black hat community which make them useless to detect backlinks - many pbn owners use htaccess to deny those bots from crawling -

Dude, not being rude but don't waste your time on this. You do not have the skills or the knowledge.
Why do you think your bots will be immune from htaccess?
If you just want data for a few sites then go ahead and write a scraper/bot/crawler/spider but if you think you can take on the big boys then it won't be a one man operation.
If you want to create a tool without significant investment then use the API's available.

competitor ranking keywords ?

seo-limited

Registered Member

Wilson Grant Fisk

Elite Member

rank bank

Regular Member

netmoney1

Elite Member

sinking50s

Newbie

seo-limited

Registered Member

KHer0

Elite Member

Wilson Grant Fisk

Elite Member

Main Menu

Marketplace

Making Money

BlackHat World