Ahrefs vs. Majestic RDs

[AngelSeo]

LooL this guy and his script. I blocked him sometimes ago after such pointless argument and now was like WTF to whom those people talk. Yeah you can safely ignore whatever he says - he have no clue whatsoever.

 

[Purush]

can't even imagine such situation when we talking about customers.
And this is why, I am learning from the most trusted source - from Google.

Then y r u here. Teasing others ?

 

[FatBee]

Yeah, your SEO experts are the most successful when it coming about link building:




Can you recommend more experts to follow ? Great entertainment.
Seems, you bet on lame horses

All of them build Expired Web 2.0s same as you, because simply you don't have $$ for solid PBNs.
Sad but true.

Also @FatBee promises in Dec 2015 that "i will invest in PBN soon as i earn some money" - seems, no luck until today.
Because he still using very low effective Web 2.0s.

Cheers, Greg.

i award you with Rank #1 Internet Warrior.

you run out of content so you start digging old posts, so you can better understand me?

Does google evolved since 2015? Yes.

Is there any public info about what i did since than? No.

 

[GregFromMoonsy]

@Stas Va

Good that you know the difference between historic and fresh index.
For any new site which is under 3 months in age - historic index is completely useless - I am agree here.

For any older site you don't see in fresh index backlinks which was discovered 4 months ago and not crawled again in last 3 months.
Such situations are the most common, as the result you not see significantly part of backlinks in fresh index.

Because Majestic have 50x larger pages index than Ahrefs, there is not possible to crawl all those pages in 3 months.
For this reason Ahrefs have more backlinks in fresh index, but much less backlinks when we will sum fresh and historic index.

And this is why professionals use historic index and choose Majestic.


Let's we talk now about crawling system.

By poor written scripts you aren't able to effective crawl the web.
I'll explain you how working my system, it will be easier to you understand the point.

So, before any single host/page are crawled - for all the hosts system will get the IP for each one.
As the next step base on IP/urls - queue being created in the way, that it not will overwhelm any single server/host.

In crawling process, information about single host speed / http code - modifying the queue.
For example, when the host replying slowly - the next request is made after longer time, same as doing it Google.

Also when some host trying to block my crawlers - some other algorithms are used (other server location, proxy,etc)
Mentioned recalcitrant host is crawled until successful crawl, then used successful method is stored for future use.

So, there no any chance to hide anything as some amateurs think - in reality they're not able to block nothing
In that simple way my system solving all crawling problems.

However, due to multi server application - it's not easy to create it.
Because it need to modify the queue "in the air" on multi servers in multi locations.

Cheers, Greg.

 

[Stas Va]

@Stas Va

Good that you know the difference between historic and fresh index.
For any new site which is under 3 months in age - historic index is completely useless - I am agree here.

For any older site you don't see in fresh index backlinks which was discovered 4 months ago and not crawled again in last 3 months.
Such situations are the most common, as the result you not see significantly part of backlinks in fresh index.

Because Majestic have 50x larger pages index than Ahrefs, there is not possible to crawl all those pages in 3 months.
For this reason Ahrefs have more backlinks in fresh index, but much less backlinks when we will sum fresh and historic index.

And this is why professionals use historic index and choose Majestic.


Let's we talk now about crawling system.

By poor written scripts you aren't able to effective crawl the web.
I'll explain you how working my system, it will be easier to you understand the point.

So, before any single host/page are crawled - for all the hosts system will get the IP for each one.
As the next step base on IP/urls - queue being created in the way, that it not will overwhelm any single server/host.

In crawling process, information about single host speed / http code - modifying the queue.
For example, when the host replying slowly - the next request is made after longer time, same as doing it Google.

Also when some host trying to block my crawlers - some other algorithms are used (other server location, proxy,etc)
Mentioned recalcitrant host is crawled until successful crawl, then used successful method is stored for future use.

So, there no any chance to hide anything as some amateurs think - in reality they're not able to block nothing
In that simple way my system solving all crawling problems.

However, due to multi server application - it's not easy to create it.
Because it need to modify the queue "in the air" on multi servers in multi locations.

Cheers, Greg.

1. Again ! my accusation to Majestic - was not the size of the database. Relevancy is not about size. Database is updating not good enough (comparing with ahrefs and Semrush). Historical Index should be use only to understand backlink profile changes. It technically can't be relevant.


2. No sense. Servers are crushing because of to much queries to the server in a short period of time. Server and site scripts of the referral can't handle it and crushes (usually 503). No difference what IP and how many IP you will use, from what servers or devises/os/browser/location you will try to make requests. I am not talking about any type of blocking or captcha. So please don't change the context.
So can you explain what is the way when you are

@Stas Va
For example, when the host replying slowly - the next request is made after longer time, same as doing it Google.

So that's already says that you can't crawl thousands of backlinks in minute without loosing in data. Even if you will create a smart queue of requests, and will manage to check links from the same domain in distance proportional to each other. That won't resolve your problem with the performance that you declare.
Google-bots are working completely different.

@Stas Va
By poor written scripts you aren't able to effective crawl the web.

Wait a minute. Google, ahrefs, Majestic and Semrush - are crawling the web. Not you. That's a completely different approach and concept. Your task as a SEO expert/marketer/employe is to crawl a list of backlinks and to audit them. For example (web tools . de) (sorry can't add links yet) is your client.

You need to check backlinks to their domain. You can use data from Google Search Console (if you have access) or get data using API from Majestic, ahrefs, Semrush or other data providers. Let's say we are using your favourite Majestic.

Lets' check overview:
imgur . com / a / j U B G M q p - great citation flow vs trust flow. 96569 k backlinks and just 20 referrals (fresh index) and 1000000 backlinks in (Historic index)

So your script will check 90000 backlinks from 20 referrals and you declare your script will do it in minutes, without loosing data. So let's check referrals.

imgur . com / f 1 N Q t k Z

What a surprise, 96456 are from 1 domain (99,89 % of referrals) so your script will make a query every few miliseconds to this domain. How you will create querue in this case ? Even if there was 20 % from the same domain, you will need serious time holds to avoid cases when your script crush the site.


Additional: Back to Majestic DATABASE disput.
Great notice for you will be that this referral is dead for a long time period. But not for Majestic fresh index. That what I mean when am telling that Majestic data is less relevant. And such error affects all majestic metrics...

 

[GregFromMoonsy]

@Stas Va

1) Check this thread, then you'll see how poor is Ahrefs Fresh index.
https://www.blackhatworld.com/seo/h...ed-1-without-an-h1-tag.1166662/#post-12492942

2) If you know how to create and update in air the quene (what I've explained to you) nothing will be crushes.
Seems, you not understood what I wrote. Because I not wrote about your IP but crawled host IP.
Also scripting language is not the way to write such effective crawlers.

3) The point is to crawl thousands IP's at the same time, not the thousands urls at the same host and by this way, noting not will be crashed.
I am write this from my expirience not any theories.

4) In partial true. You are not able to index thousands of urls from the single poor host in short time. (see point 2,3)
I am don't care about poor links from low TF domains, because they have tiny power.
Especially as you wrote, from the same domain.

Cheers, Greg.

 

[Ely]

Hei @GregFromMoonsy , thanks! Tried Majestic (used ahrefs for a long time) today and found a lot of hidden backlinks from my competitors. It's amazing! Thanks again dude!

 

[Stas Va]

1.Just numbers, same issue you made giving numbers about database size ignoring data relevance.
Garbage data - useless data. Same arguments, I already destroyed with exact case.

@Stas Va

2) If you know how to create and update in air the quene (what I've explained to you) nothing will be crushes.
Seems, you not understood what I wrote. Because I not wrote about your IP but crawled host IP.
Also scripting language is not the way to write such effective crawlers.

@Stas Va
3) The point is to crawl thousands IP's at the same time, not the thousands urls at the same host and by this way, noting not will be crashed.
I am write this from my expirience not any theories.

Can you explain what's info about backlink you will get crawling IP ? How your crawler will eve check if the link is really there or ahrefs attributes ?

@Stas Va
4) In partial true. You are not able to index thousands of urls from the single poor host in short time. (see point 2,3)
I am don't care about poor links from low TF domains, because they have tiny power.
Especially as you wrote, from the same domain.

Cheers, Greg.

Are u sure now ? Because few posts earlier you told me ...

You are wrong even double.

Not true, except if you using poor written scripts.

Cheers, Greg.

And that's the punch line.

 

[GregFromMoonsy]

Hei @GregFromMoonsy , thanks! Tried Majestic (used ahrefs for a long time) today and found a lot of hidden backlinks from my competitors. It's amazing! Thanks again dude!

Thank you Ely.
I am glad that you discovered Majestic to better analyze competitors than with Ahrefs.

As the next step you should start build the links with right Topic, learn more here:
https://www.blackhatworld.com/seo/ahrefs-vs-majestic-rds.1165105/page-3#post-12488647


@Stas Va

before any single host/page are crawled - for all the hosts system will get the IP for each one.
As the next step base on IP/urls - queue being created in the way, that it not will overwhelm any single server/host.

what's info about backlink you will get crawling IP ?

At first - you should throw your team away, because they aren't able to use the brain at all.
As the next step, you must post the request in "Hire a Freelancer" section with the request to create mentioned solution.

I am pretty sure that each clever programmer not will be have any hassle with understand what I have on mind with mentioned sentences.
Especially if he have the real experience in web crawling.


Let's we talk about the next thing related to Majestic and Ahrefs index:

Garbage data ... ignoring data relevance...

Tell me the domain name which have as you said the "garbage data" in Majestic History Index.
I will destroy your fake story in few minutes, because I am pretty sure that you created your account yesturday only for trolling reason with Ahrefs.

Seems @AngelSeo also only trolling in this thread. Maybe your nick it's just his second account on BHW.
He know everything but not understand absolutely nothing same as you.

As I previously said, Majestic crashing Ahrefs many times.
Typical example with the fake theory about bigger Ahrefs fresh index:


https://www.blackhatworld.com/seo/h...ed-1-without-an-h1-tag.1166662/#post-12492942


So I am not surprised by @Ely post:

Tried Majestic (used ahrefs for a long time) today and found a lot of hidden backlinks from my competitors. It's amazing! Thanks again dude!

Cheers, Greg.

 

[b1step-ahead]

Can someone explain why Ahrefs and Majestic Rds don't match? Always Ahrefs says like 500 RD but majestic says only 10-20 RDs. Anyone know?

Just get both and you will be good

 

[HustleTong]

Ahrefs is much faster and gives more professional overview.

 

[Mr Positive]

Ahrefs is much faster and gives more professional overview.

Ahrefs is good but I like moz too both can give deep analysis and are helpful when you’re trying to dig deep
Into competitors

 

[Stas Va]

At first - you should throw your team away, because they aren't able to use the brain at all.
As the next step, you must post the request in "Hire a Freelancer" section with the request to create mentioned solution.

I am pretty sure that each clever programmer not will be have any hassle with understand what I have on mind with mentioned sentences.
Especially if he have the real experience in web crawling.

You didn't answer. There is a CLEAR QUESTION ? Who cares what I should say to my team ? What data your crawling system get by Host IP. How your system check if the link even exists on the page, without crawling page ?
Please give me an example what data your tool is crawling. Can you add a screen ?!

Let's we talk about the next thing related to Majestic and Ahrefs index:


Tell me the domain name which have as you said the "garbage data" in Majestic History Index.
I will destroy your fake story in few minutes, because I am pretty sure that you created your account yesturday only for trolling reason with Ahrefs.

Seems @AngelSeo also only trolling in this thread. Maybe your nick it's just his second account on BHW.
He know everything but not understand absolutely nothing same as you.

As I previously said, Majestic crashing Ahrefs many times.
Typical example with the fake theory about bigger Ahrefs fresh index:

I will choose site you used earlier.

Historical DATA:
Let's check 20 biggest referrals from Majestic.

Oh... checking only first 23 referrals we found out that 318443 - does not exist already on the site with 1 mln backlinks in historical. Lets BET I will find at least 500000 backlinks that doesn't exist. Whats their data size says about that.

But OK. Everybody knows that for exactly our goal (to check your backlinks) historical index - is useless.
Let's check referrals from fresh index.

- 2532 (> 7 %) backlinks just after first look
If you will continue to speculate with Majestic database size and their numbers. I will check all backlinks from this website.
Because my crawling system really exist, and I can check them all. And I am sure that I will find at least 20 % of Majestic Fresh Database, doesn't works and exists anymore. And doesn't exist for a time period much more then 3-4 months.

So I am not surprised by @Ely post:


Cheers, Greg.

That's absolutely don't boder me. How authoritative opinion of the man who just used Majestic for the first time.

I do use Majestic, and I also use ahrefs. As I mentioned earlier Majestic do have interesting metrics like TF & CF, that gives you opportunity to understand the ration and so on.
But only people who didn't work enough with both tools, can deny, that ahrefs database is much cleaner and relevant. Ahrefs & Semrush are great tools for any lvl.

 

[GregFromMoonsy]

You didn't answer... crawling system get by Host IP

I told you, if they (your team) aren't able to understand for which reason they need to check IP when we talking about creating the quene, they don't use the brain at all, because here I haven't on mind "crawling the IP". For this reason - my suggestion to you was: hire the clever programmer who don't will be have any hassle with understand this part.

Historical DATA ... 318443 - does not exist ...only people who didn't work enough ...historical index - is useless

Only what is useless here - it's your (and your "team") very low knowledge about crawling the web.
And bellow you'll find why.

At first and most important: if you want to crawl a chinese sites (on chinese DNS and/or hosted in China) you should have for this task crawling servers located in mainland China - this is why I am said that for crawling purpose you need servers in many locations.

It's because they blocking a lot of traffic outside the China at three levels - starting at great China firewall, dns and servers level.
In this case - all your theories and numbers (related to example with Majestic screenshot) are absolutely worth nothing.

So please show me the example with Majestic Historical index and many dead links for any domain which haven't links from China.
Then we can continue to talk.

Cheers, Greg.

 

[Stas Va]

Everything start on the point: "Usually this process (crawling backlinks pages) crush lots of referrals on cheap servers.

Not true, except if you using poor written scripts.

Cheers, Greg.

The point is to crawl thousands IP's at the same time, not the thousands urls at the same host and by this way, noting not will be crashed.
I am write this from my experience not any theories.

I told you, if they (your team) aren't able to understand for which reason they need to check IP when we talking about creating the quene, they don't use the brain at all, because here I haven't on mind "crawling IP". For this reason - my suggestion to you was hire the
Only what is useless here - it's your (and your "team") very low knowledge about crawling the web.
And bellow you'll find why.

Cheers, Greg.

I told you, if they (your team) aren't able to understand for which reason they need to check IP when we talking about creating the quene, they don't use the brain at all, because here I haven't on mind "crawling IP". For this reason - my suggestion to you was hire the
Only what is useless here - it's your (and your "team") very low knowledge about crawling the web.
And bellow you'll find why.

Cheers, Greg.

And my question was how your quene allows you not to create request to the site with performance and quality that you have declared. You didn't answer it and I was using your IP quene to troll you. That's don't fix the problem. And to create smart quene of requests and filter them by domains, subnets and so on, was like obvious. On and real site example I demonstrated that's with the perfomance you declared, almost for any average domain you will crush few sites checking backlinks. And actually not always the sites (in most cases, but not always) will be total crap, domains can be good. For example domains with redirects can set on a crappy free hosting and so on. Cases can be very different. You initially declared that GREAT DEVS an your GREAT SCRIPT don't have such problems.

4) In partial true. You are not able to index thousands of urls from the single poor host in short time. (see point 2,3)
I am don't care about poor links from low TF domains, because they have tiny power.
Especially as you wrote, from the same domain.

Cheers, Greg.

So me and my team wasn't right ? What you mean saying "partial" true ?
You have solution ?. Smart Query is obvious "partially" solution, because it's don't fix it fully.
It can reduce risks using more efficient and uniform load distribution. But not fix, the problem. In most cases >50 % of all backlinks are from 10 % of referrals.
So that's a very frequent issue.

Let's go Majestic Historical Data now:
Let's talk simple local site, that is ranking by "flowers delivery" in miami - https://www.kingscreekflowersandgifts.com

To make it fair I filtered all deleted backlinks (but actually they are considered in statistic you use).
If you will delete few duplicats , you will see only.

200 backlinks 66,4 % doesn't exist, and you get page 404.

BUT LET's CHECK IF BACKLINKS TO kingscreekflowersandgifts.com REALY ARE SET ON REMAINED PAGES.

You can check all backlink status manually.
DETAILS FOR ALL BACKLINKS: https://docs.google.com/spreadsheets/d/1OrkMbwhpl1nyphj7L8z7JvOY4saDfEqxtFPPHZ4Zpj4/edit#gid=0

 

[GregFromMoonsy]

You didn't answer

I told you (but you still can't understand), first you need the IP address for each host to create mentioned queue.
Without the IP address you aren't able to create the effective queue because:

1. You don't know which domains are on the same IP or /29 range - in result you crashing single server or you receiving 403 50x http code.
2. You don't know in which country server is located, in result crawling is slow or even impossible (such mentioned China)
3. You trying to crawl the domains which even don't have the DNS A,AAAA,CNAME records or domain was deleted, in result crawling have very low effectivity.
4. You must have daily updated domains database with current domain status (such active,expired,deleted,redemption etc), because in other case - you crawling suspended pages etc (where in result you're not able to find the link)

As I told you too - quene is modificated in the air - basing on crawling stats, such http code / page speed.
If you receiving 403 50x http code - you should run additional algorithms to detect if IP was banned or just server have some issues.

So, you already learned from me a lot of things without even simple "thanks", which cost you nothing.
Instead of this, you trolling with quoting my sentences, which in reality you aren't able to understand technically.

What I already learned from you ?
Technically nothing, except that some people are not worth any help because they already know everything but don't understand nothing.

Cheers, Greg.

 

[Stas Va]

Great that our other dispute about the senselessness of the statistics of crawled pages, which "majestic" declare and you have so actively used in this thread was closed.

I told you (but you still can't understand), first you need the IP address for each host to create mentioned queue.
Without the IP address you aren't able to create the effective queue because:

1. You don't know which domains are on the same IP or /29 range - in result you crashing single server or you receiving 403 50x http code.
2. You don't know in which country server is located, in result crawling is slow or even impossible (such mentioned China)
3. You trying to crawl the domains which even don't have the DNS A,AAAA,CNAME records or domain was deleted, in result crawling have very low effectivity.
4. You must have daily updated domains database with current domain status (such active,expired,deleted,redemption etc), because in other case - you crawling suspended pages etc (where in result you're not able to find the link)

As I told you too - quene is modificated in the air - basing on crawling stats, such http code / page speed.
If you receiving 403 50x http code - you should run additional algorithms to detect if IP was banned or just server have some issues.

3 from 4 of your points are not relevant to our discussion. Our disput is related only with the first point. I already showed you that most of backlinks for almost every average site, are from 10 % of referrals domains. You already acknowledged that queue will not resolve the original problem .

In partial true. You are not able to index thousands of urls from the single poor host in short time. (see point 2,3)
Especially as you wrote, from the same domain.

If we will keep in mind the performance we are talking about.

However, my crawling system is able to check fat thousands urls in few minutes and check each link.
So, for professionals it's not the problem at all.

So "PRO" why do you steel publish irrelevant commonplace technical things how to create queries on factors that are not related to our problem, if that don't resolve the initial problem. I thought it's not the problem at all.

How DOMAIN DNS status matters, if we are talking about working domains with sites whose server crashes upon requests ... Which seriously limits either the performance or the quality and quantity of data. You wrote so much, but you wrote nothing.

So, you already learned from me a lot of things without even simple "thanks", which cost you nothing.
Instead of this, you trolling with quoting my sentences, which in reality you aren't able to understand technically.

I'm trolling you, because it funny to see how you try to look more then you are. That's guru role you are trying to play is hilarious. Thanks.

ps.
You can destroy me any time, just by showing how "YOUR CRAWLING SYSTEM" in work and report/excell/csv what exactly it crawls. I think that will be an amazing advertisement of your services/tools and so on.

 

[GregFromMoonsy]

Great that our other dispute about the senselessness of the statistics of crawled pages, which "majestic" declare and you have so actively used in this thread was closed.

Ask Ahrefs how many servers they have in China with their "biggest" index
I'll tell you - they using only OVH, AFAIK OVH don't have any single server in China and each amateur is able to block Ahrefs crawlers at iptables level in invisible way.

You can destroy me any time...

Please show me the screenshot of Ahrefs Historical Index + Google sheet with dead BL for this domain, I guess I'll be laugh strong.
That will be enough to show reality.

Hope that your amazing crawling script not need the whole week for crawl those few urls from Ahrefs.
As it was designed by team of first class engineers who asking on forum about help

Cheers, Greg.

 

[Stas Va]

Ask Ahrefs how many servers they have in China with their "biggest" index
I'll tell you - they using only OVH, AFAIK OVH don't have any single server in China and each amateur is able to block Ahrefs crawlers at iptables level in invisible way.

Please show me the screenshot of Ahrefs Historical Index + Google sheet with dead BL for this domain, I guess I'll be laugh strong.
That will be enough to show reality.

Sure. The result will be less cleaner, because I actually can't filter deleted/removed backlinks backlinks as we did in Majestic (weird, that's there no so option). But not a problem, the most important is the final report. Who will have more active backlinks that really exists.

Moreover to be fair, almost 600 backlinks from 1598 - where duplicates. So overall there are only 977 unique URL's.
Almost a half are 404 (but keep in mind, we can't filter deleted links, how we made in Majestic).

And the final result is

So what we have ...
90 (AHREFS) vs 61 (MAJESTIC) (32 % difference).
AHREFS WIN

All the details: https://docs.google.com/spreadsheets/d/1QfjHg5pnSimOH3p2_mMph4ULkhAfn3uk0rrK68yvJVo/edit?usp=sharing

Still want to argue with me ? So let's then check another website. But now, you will provide proofs. It should be kinda easy for your script...

Hope that your amazing crawling script not need the whole week for crawl those few urls from Ahrefs.
As it was designed by team of first class engineers who asking on forum about help

Cheers, Greg.

1. I didn't ask help. I was wondering how exactly you resolved exact problem, because you declared you resolved it. But after few irrelevant posts you just accepted, that the issue can't be fully fixed.

Please don't blame me on this. I new here, so can't differ professionals from talkings heads just by nickname yet.

2. I steel don't see any sign of the existence of your puffy script. After all, it’s not me who spoke in full about his "script" before even somebody ask about it. You just decided that it would be cool to say about a cool technical thing that you don’t have.

@FatBee

Maybe you not finished any school and for this reason it's very hard to you compare really big numbers such:

8 485 618 151 026 Majestic pages in index
149 000 000 000 Ahrefs pages in index

Cheers, Greg.

Also can you please tell me, do you feel difference between "pages in index" and "url found"...
Do you understand the concept behind URL ?


For example:
http://site.com/page site.com/page
https://site.com/page?utm_content=b...l&utm_source=facebook.com&utm_campaign=buffer
https://site.com/page?utm_content=b..._source=linkedin.com&utm_campaign=GregIsWrong
https://site.com/page

Are those pages different, and what you will tell about URLs ?

Please share your "professional" experience...
What you can tell about internal links, .... do they also have URLs ?

WHY I'M ASKING ? Because it's interesting why "professional" like you compares two absolute different concepts?

You even didn't use statics about "crawled" URLs. You decided to choose the biggest number in Majestic statistics, and compare with the lowest in Ahrefs statistic (except domains).
Do you see the difference between "crawled" and "found" ? Great technique for slippy salesman.

I think fake Mythbusters for our industry is way much more dangerous than the myths themselves.
Professional ? Stop trying to seem like, just be one.

 

[GregFromMoonsy]

@Stas Va

I see on your Ahrefs screenshot 1,598 backlinks, but in Google sheet you listed ONLY 978
Can you update it please to full list ?
I don't like such manipulations.

When you'll update the Google sheet with Ahrefs BL - I'll run my crawlers to verify all BL.
Seems, your Ahrefs have bigger chance to win, because Majestic have "only" 1,269 backlinks.

I'll check the raw BL data without any filter, as you aren't able to use such filter in Ahrefs.
And I don't like such manipulations as I said previously.

Then we'll compare my results with yours, to see the reality.
Hope that you will not disappear in mean time

Just be the man.
And we'll continue.

Cheers, Greg.