[small case study] Semrush keyword difficulty metric deserves more credit that it is given

I have always held semrush in high regards when it comes to keyword research tools. I have this blind belief that semrush knows more about google's SERP algorithms than any other *cough* Ahrefs *cough* premium expensive tools.
The keyword difficulty metric of Semrush felt extremely deceptive at first glance. The keywords listed as "very easy" in semrush always have high competition in SERPs when you manually check them.

So decided to blindly follow SEMrush's keyword difficulty advice for a batch of keywords on my new test blog. I filtered out around 15 keywords having KD of 0/1. Although as per KD I can rank immediately even with a new site for those keywords, manually checking the SERPs showed a different scenario. I went with my little endeavour regardless and wrote separate articles dedicated to each keyword and slowly published them last week. I had no hope for ranking for any of those keywords even in the first page which was dominated by DA 30-40 sites.

Fast forward to today : Every fucking keyword in that batch I wrote an article on is ranking in top 3 or top 10. Below are some of the keyword rankings (ignore the ones with KD >1) and actually getting clicks as per GSC.

I actually ranked my new test site for those keywords with the power of articles alone by displacing literal authority sites in the SERPs exactly as Semrush predicted.
This was a small scale case study but for me personally this just boosted my trust for Semrush KD metric. Semrush definitely knows something about keyword ranking factors on google that cannot be explained by manual analysis of SERPs. I was a big fan of checking SERPs manually to determine whether to write for a keyword or not. Not anymore. I will be blindly following Semrush KD metric.

can you update this thread in 2-3 weeks? If you still rank after 3 weeks something changed at google (again).

But awesome results nonetheless

Valuable, will have to try it out for myself as well

tazarbm said:

can you update this thread in 2-3 weeks? If you still rank after 3 weeks something changed at google (again).

But awesome results nonetheless

Click to expand...

Absolutely.
But I do feel like this is not a part of Google dance.

tattooedbuddha said:

Valuable, will have to try it out for myself as well

Click to expand...

Do share your inference too if possible

Waifu_tatyana said:

I have always held semrush in high regards when it comes to keyword research tools. I have this blind belief that semrush knows more about google's SERP algorithms than any other *cough* Ahrefs *cough* premium expensive tools.
The keyword difficulty metric of Semrush felt extremely deceptive at first glance. The keywords listed as "very easy" in semrush always have high competition in SERPs when you manually check them.

So decided to blindly follow SEMrush's keyword difficulty advice for a batch of keywords on my new test blog. I filtered out around 15 keywords having KD of 0/1. Although as per KD I can rank immediately even with a new site for those keywords, manually checking the SERPs showed a different scenario. I went with my little endeavour regardless and wrote separate articles dedicated to each keyword and slowly published them last week. I had no hope for ranking for any of those keywords even in the first page which was dominated by DA 30-40 sites.

Fast forward to today : Every fucking keyword in that batch I wrote an article on is ranking in top 3 or top 10. Below are some of the keyword rankings (ignore the ones with KD >1)
View attachment 254461View attachment 254460
I actually ranked my new test site for those keywords with the power of articles alone by displacing literal authority sites in the SERPs exactly as Semrush predicted.
This was a small scale case study but for me personally this just boosted my trust for Semrush KD metric. Semrush definitely knows something about keyword ranking factors on google that cannot be explained by manual analysis of SERPs. I was a big fan of checking SERPs manually to determine whether to write for a keyword or not. Not anymore. I will be blindly following Semrush KD metric.

Click to expand...

Doesn't work every time, but it happens. Give it a while to see how you main the position.

For the most part, I think semrush is decent with keywords but backlinks is where they flop terribly.

HenryObi said:

Doesn't work every time, but it happens. Give it a while to see how you main the position.

For the most part, I think semrush is decent with keywords but backlinks is where they flop terribly.

Click to expand...

Maybe I have confirmation bias but still I do feel like this is something.
My main site (which is fairly new too) where my entire keyword research was based on manual SERP analysis none of my articles are seeing any traction at all. Not even the zero volume keywords where SERPs are dominated by user generated content and theoretically I have huge chances of ranking in top 3. In fact that site didn't even experience any prominent Google dance at all (unless you consider slow de indexation of perfectly fine articles as google dance )

While this 1 month old test site has witnessed so much activity this early in the game.

Waifu_tatyana said:

Absolutely.
But I do feel like this is not a part of Google dance.

Click to expand...

if you're not bouncing in and out of the SERPs multiple times a day it's not a dance

Impressive stuff indeed.

Waifu_tatyana said:

Maybe I have confirmation bias but still I do feel like this is something.

Click to expand...

Of course it is something.

Waifu_tatyana said:

My main site (which is fairly new too) where my entire keyword research was based on manual SERP analysis none of my articles are seeing any traction at all. Not even the zero volume keywords where SERPs are dominated by user generated content and theoretically I have huge chances of ranking in top 3. In fact that site didn't even experience any prominent Google dance at all (unless you consider slow de indexation of perfectly fine articles as google dance )

Click to expand...

Maybe you should focus on using semrush on your main site.

The problem with any tools metrics is that it's not always 100% correct.
You are fascinated just by a simple case study which is the case with most of us.
It has happened with me too and at first I was in awe of so many tools and metrics.
But once you keep doing that and get addicted to trusting these metrics blindly.
And then it ditches you in some of your important projects, too bad if that's a client project.
This is where issue becomes headache.
So, if someone likes to play safe it's better to check the serp results once manually.

BTW Semrush is a much better keyword suggestion tool compared to others just like Ahrefs which is a better backlink research tool.

Good share.
This will be helpful for whom asking for - which is the best tool to know keyword Density/ ahrefs or semrush/ so and so!

HenryObi said:

Maybe you should focus on using semrush on your main site

Click to expand...

That's the plan for now
Currently creating a batch of keywords for my main site solely based on KD metric

MKSKS said:

The problem with any tools metrics is that it's not always 100% correct.
You are fascinated just by a simple case study which is the case with most of us.
It has happened with me too and at first I was in awe of so many tools and metrics.
But once you keep doing that and get addicted to trusting these metrics blindly.
And then it ditches you in some of your important projects, too bad if that's a client project.
This is where issue becomes headache.
So, if someone likes to play safe it's better to check the serp results once manually.

BTW Semrush is a much better keyword suggestion tool compared to others just like Ahrefs which is a better backlink research tool.

Click to expand...

Yeah I will be writing articles on keywords I dug from manual SERP analysis too. But I won't be neglecting the low KD keyword suggestions from Semrush too no matter how deceptive they look. Personally, I believe each and every article adds some value to the site , even the high competition ones.

dragonlube said:

Good share.
This will be helpful for whom asking for - which is the best tool to know keyword Density/ ahrefs or semrush/ so and so!

Click to expand...

Thanks

Waifu_tatyana said:

I have always held semrush in high regards when it comes to keyword research tools. I have this blind belief that semrush knows more about google's SERP algorithms than any other *cough* Ahrefs *cough* premium expensive tools.
The keyword difficulty metric of Semrush felt extremely deceptive at first glance. The keywords listed as "very easy" in semrush always have high competition in SERPs when you manually check them.

So decided to blindly follow SEMrush's keyword difficulty advice for a batch of keywords on my new test blog. I filtered out around 15 keywords having KD of 0/1. Although as per KD I can rank immediately even with a new site for those keywords, manually checking the SERPs showed a different scenario. I went with my little endeavour regardless and wrote separate articles dedicated to each keyword and slowly published them last week. I had no hope for ranking for any of those keywords even in the first page which was dominated by DA 30-40 sites.

Fast forward to today : Every fucking keyword in that batch I wrote an article on is ranking in top 3 or top 10. Below are some of the keyword rankings (ignore the ones with KD >1) and actually getting clicks as per GSC.
View attachment 254461View attachment 254460
I actually ranked my new test site for those keywords with the power of articles alone by displacing literal authority sites in the SERPs exactly as Semrush predicted.
This was a small scale case study but for me personally this just boosted my trust for Semrush KD metric. Semrush definitely knows something about keyword ranking factors on google that cannot be explained by manual analysis of SERPs. I was a big fan of checking SERPs manually to determine whether to write for a keyword or not. Not anymore. I will be blindly following Semrush KD metric.

Click to expand...

Thanks for sharing this information here, even i was planning to use semrush keyword research tool now but is there any way get extra discount

Waifu_tatyana said:

The keywords listed as "very easy" in semrush always have high competition in SERPs when you manually check them.

Click to expand...

Can you share an overview of what kind of pages were ranking in serp?
Or
How you determined that the serp results were showing highly competitive niche/keyword?

MKSKS said:

Can you share an overview of what kind of pages were ranking in serp?
Or
How you determined that the serp results were showing highly competitive niche/keyword?

Click to expand...

Sure. Will share the screenshots after I get back home.

For manual SERP analysis I use Moz metrics. For some keywords notable organizations in my niche was ranking.
For 4 keywords my DA 2 site has actually beaten the organisation sites in position.

Waifu_tatyana said:

Sure. Will share the screenshots after I get back home.

For manual SERP analysis I use Moz metrics. For some keywords notable organizations in my niche was ranking.
For 4 keywords my DA 2 site has actually beaten the organisation sites in position.

Click to expand...

SERP analysis doesn't necessarily require any tool, it's good to have access to any SEO tool for this purpose though.
A simple google search using browser plugins like SEOQuake and a free Semrush account is enough.

By serp analysis I meant what type of websites are ranking in the serp results?
Like are there any UGC websites like social media, forums, online communities.
What's the overall intent of these ranking pages?
Are there any non-niche relevant or unconventional websites there something like a parasite property.
Are there any YouTube videos in the serp results if so how many.
What about the featured snippets or other serp features in these results?

Waifu_tatyana said:

Maybe I have confirmation bias but still I do feel like this is something.
My main site (which is fairly new too) where my entire keyword research was based on manual SERP analysis none of my articles are seeing any traction at all. Not even the zero volume keywords where SERPs are dominated by user generated content and theoretically I have huge chances of ranking in top 3. In fact that site didn't even experience any prominent Google dance at all (unless you consider slow de indexation of perfectly fine articles as google dance )

While this 1 month old test site has witnessed so much activity this early in the game.

Click to expand...

Following on this. Very interesting.
I have recently subscribed to SEMRush and paid the full price for 1,500 kw, even though after filtering them one website only needed 500 something kws (which is the only one I'm working on at this time), this thing is bloody expensive! but I can't trust these shared SEMRush. I can possibly share mine with a few trusted people but thats it.
I have filtered the low difficulty KW today so I can start working on them. Your post attracted my attention because this what I'm working on now.
My question are:
1- What do you do with the "manual SERP analysis" as I don't have much experience in this. Is it important when the kw difficulty is between 0 to lets say 8?
2- How did you write your articles, did you use SEMRush service? or did you write them yourself using SEMRush SEO content template? I have used the content template on two pages for 2 kws this past weekend and waiting to see if there are any improvement.
3- I'm wondering if their SEO content template One Page SEO checker are any good?

AAW said:

1- What do you do with the "manual SERP analysis" as I don't have much experience in this. Is it important when the kw difficulty is between 0 to lets say 8?

Click to expand...

Manually checking the first page of Google for the keyword I am trying to write an article for to assess the competition level. There are several YT tutorials on this.

AAW said:

2- How did you write your articles, did you use SEMRush service? or did you write them yourself using SEMRush SEO content template? I have used the content template on two pages for 2 kws this past weekend and waiting to see if there are any improvement.

Click to expand...

My articles were manually written by me with some extra key points generated by Chat GPT which I feel are lacking in the content of sites already ranking. Google loves new information (I have seen a site simply posting a spun version of another site's content, adding a few extra lines of seemingly useful information at the end and even ranking above the site it copied from)

AAW said:

3- I'm wondering if their SEO content template One Page SEO checker are any good?

Click to expand...

I used to care at the beginning about perfecting that Rankmath SEO score but now I don't. These metrics based on which these SEO tools assign a score to your content are outdated (for example you don't need to have the exact match keyword even once in your content. You can simply write naturally and Google is smart enough to pick up the context of your site). So these so-called SEO checkers like Surfer SEO and Rankmath are useless. The only plugin I use while writing my content is Grammarly to help me correct the subtle grammatical errors in my content.

MKSKS said:

By serp analysis I meant what type of websites are ranking in the serp results?

Click to expand...

Can't reveal too much without giving away my niche. But let's put it like this: for some keywords, the SERPs had long-standing authority sites on the first page including a well-known certification agency in my niche.

MKSKS said:

Like are there any UGC websites like social media, forums, online communities.

Click to expand...

I have uncovered these extremely low-comp keywords outside of Semrush using the alphabet soup method. Weirdly enough my articles aren't performing well for quite a few of these keywords even though the top results are almost plagued by non-niche forum UGC.

MKSKS said:

Are there any non-niche relevant or unconventional websites there something like a parasite property.

Click to expand...

For the keywords suggested by Semrush? None. If you checked the SERPs by yourself you'd surely have excluded the keyword.

AAW said:

How did you write your articles, did you use SEMRush service?

Click to expand...

Manually. I know a bit about my niche so I don't have any problem writing long-form content by myself. I did take a look at the content of the ranking sites for suggestions. I don't like these on-page SEO score-assigning tools as I feel they are outdated.