[Ultimate Guide] How to Analyse SERP and Keyword Competition

affbull said:

Thanks. Is there a wordpress theme you would recommend for this purpose?

Click to expand...

Holzr said:

I use generatepress, it's lightweight and quick which is great for SEO.

Click to expand...

goflippa said:

What about Rehub theme ? Is it good for startup?

Click to expand...

Holzr said:

Edit: Some of the pictures seem to be broken for me, can you let me know in the comments if they are for you and I can see about uploading them to somewhere like Imgur and updating the images on this guide.

Does this sound like you:

1. Find keyword on keyword research tools such as Ahrefs or KWFinder
2. Filter for low scores/"easy" keywords
3. Write an article
4. Wait for the $$$ to roll in
5. The keyword doesn't rank

Unfortunately, this is the approach many people take when doing keyword research, it's what the guides and many gurus say you should do, even the tools say the keyword should be "easy" to rank for.

In this article, I'm going to go over some of the steps I take when evaluating a new keyword I'd like to rank for either in a https://www.blackhatworld.com/seo/1-002-posts-special-simple-way-to-find-profitable-amazon-affiliate-niches.1054603/ or one I'm already in. Keyword research is an art form and science and unfortunately, there's no easy way to evaluate the competition. As nice as a score would be, they're just not accurate. You'll see why I'd recommend avoiding these scores and backlink estimates as we go along in this guide...

In order to pick truly low competition keywords, you need to employ a variety of tactics. I'll go over how to analyze SERP and true keyword difficulty both for free and paid. This article is mainly aimed at affiliate terms but pretty much everything can be applied to all sorts of keywords.

This article is packed full of links to some excellent articles that I highly recommend you read if you want to improve your SEO knowledge.

I'll mostly be using Ahrefs to do a lot of the work (it's a great tool) but you can do a lot for free using other tools.

I shall be using the keyword "best running shoes" as my keyword that I shall be analyzing in this guide. It's a high volume search term and by using it I won't be outing any smaller/vulnerable sites, niches or keywords rather than going for a more niche keyword (hopefully that makes sense).

Ahrefs shows a https://ahrefs.com/keyword-difficulty (KD) of 24 (Medium) - "you’ll need backlinks from ~27 websites to rank in top 10 for this keyword"

I like to use the following when doing competition research:

• Backlink/Referring Domain count
• Allintitle results
• Relevance/Search Intent
• Content length/quality and on-page SEO
• Types of websites ranking
• SERP History

Tools

Chrome Extensions

• MozBar
• Ahrefs Bar
• SEOQuake
• SEO Minion
• Page Load Time

Other Tools

• Ahrefs
• Common Sense

I'm going to go in order of easiness for evaluating the competition and step by step (sometimes you won't need to go through them all as you'll figure out the difficulty pretty early on)

Note: I shall be talking about a fair few websites and many affiliate websites. All the ones I am discussing are big media owned websites or have been discussed on the web as case studies etc. I am not trying to out any websites.

Allintitle Results/KGR

A super easy (and free) tactic I like to use to analyze the potential difficulty of a keyword. This is the https://nichesiteproject.com/keyword-golden-ratio-best-keyword-research-rank-google/ and also the https://dumbpassiveincome.com/keyword-golden-ratio-kgr-63/.

Have a read of both articles to see how the formula works.

For results with over 250 monthly searches, you want to use the rule of 63.

Let's calculate the score for our keyword:

277,000 results

Monthly Searches: 52,000

KGR Score: 5.33

You don't even need to calculate the score to know it's not a KGR term. Especially as it's over 250 MS you would need to see 63 results max! Without even looking at any other results it's clear that this is going to be very hard to rank for. You're competing with 277,000 pages for 10 spots.

Now, some people argue against the KGR effectiveness but I believe it carries some weight. I have personally used it as an experiment for my new site and have had a lot of success with it so far (using the rule of 63 for terms over 250 MS). Take it with a pinch of salt, I like to use it as a quick way of evaluating what I'm going up against.

In my opinion, the KGR works best for long tail keywords. Why? Because long tails have a very specific search intent and if there's hardly any allintitle results it's likely that Google is serving the user with an irrelevant result to try and satisfy the search term.

A great example of good search terms is best x for x ones. Often you can find hidden gems that the SERP is not relevant and KGR works really well for these terms. Here's a screenshot for my website, it's got 106k words and is less than 3 months old. No backlinks (I've only just started building this week) - The keywords are all best x for x terms.

Note: This applies to both Allintitle and this section. It's important to check the order of words in the keyword and spelling of a term and perhaps the variance of the keyword when evaluating a keyword. E.g. best mens running shoes, best running shoes mens, best mens shoes for running and best running shoes for men. The second result and third result might not show many allintitle results or pages ranking for the exact term. At this point you might think the term is easy, however, it's important to understand the context of the keyword and how Google interprets the search query. Just because the second and third results show low results doesn't mean you can rank for it. In this example results for best running shoes for men will likely dominate the SERP. It's the same meaning as the other terms so you'll be competing with these pages also not just for the obscure order and wording of a keyword such as best mens shoes for running.

Relevance and Search Intent

This is usually the first or second tactic I use. Many years ago when Google was not as smart it couldn't understand, evaluate or interpret what the user's search intent was. Now we have RankBrain (I highly recommend reading this https://backlinko.com/google-rankbrain-seo to understand this section of the article and this tactic) Google has got a lot better of understanding and serving the user's with a more accurate SERP.

Why is RankBrain important? Well, now that SERP results are more accurate anytime you see results that show irrelvant results it's usually a good sign (you'll often find that the result has low allintitle results as well). It shows Google is struggling to show pages that will match the users' search intent.

Edit: Good examples of irrelevant results on affiliate terms are e-commerce stores like Amazon. For best x terms you often see Amazon bestseller pages rank. This is a good sign! The pages are not optimized and just rank on authority.

You'll often see this in best x for x results since these are very specific a lot of articles won't cover them or there be a header in a main best x article. Since H1, Title Tag and URL, etc play a significant part in SEO these terms are often fairly easy to rank for and therefore outrank the less optimized irrelevant results already ranking.

The best x articles that are ranking might be strong in terms of backlinks but this is one of the times you can, to a certain extent, ignore them as you are not directly competing with them (they won't be targeting your exact keyword they're targeting the best x short tail term).

Note: This applies to both Allintitle and this section. It's important to check the order of words in the keyword and spelling of a term and perhaps the variance of the keyword when evaluating a keyword. E.g. best mens running shoes, best running shoes mens, best mens shoes for running and best running shoes for men. The second result and third result might not show many allintitle results or pages ranking for the exact term. At this point you might think the term is easy, however, it's important to understand the context of the keyword and how Google interprets the search query. Just because the second and third results show low results doesn't mean you can rank for it. In this example results for best running shoes for men will likely dominate the SERP. It's the same meaning as the other terms so you'll be competing with these pages also not just for the obscure order and wording of a keyword such as best mens shoes for running.

Looking at Backlinks (Internal and External) and Referring Domains

Take a look at the number of backlinks each of the SERP results has, analyze the backlinks and work out a rough estimate of how many backlinks you'll need to rank. Looking at internal backlinks is also super important and overlooked massively. If you want to learn about the significance of internal links then I'd highly recommend reading the following two articles from Ahrefs.

• https://ahrefs.com/blog/google-pagerank/
  
• https://ahrefs.com/blog/internal-links-for-seo/

If you often wonder how some of these authority websites rank in 1 day for a term internal links are your answer. With Ahrefs you can explore the internal links pointing to a page. Many people will see SERP with <10 backlinks and think easy, however, if they have homepage links and you're trying to compete with BIG sites like top ten reviews, best products, etc. think twice.

Back on track, let's look at the SERP for our term:

So, Ahrefs reckons you'll need 27 backlinks to break into the SERP. Can you see now why you shouldn't take these scores and numbers at face value? Especially if you combine that with the high allintitle results and taking a look at beyond page 1 and beyond many of the results have quite a few backlinks, enough to make you wonder why they aren't in the SERP if Ahrefs is saying you only need 27.

At this point, I'll take a couple of sites at random on the SERP and analyze the backlink quality and relevance.

This article is going to be seriously long if I analyze more than 1 website so I'm just going to take the first result and the third result (running shoes guru)...

The first result (http://www.runnersworld.com/gear/a19663621/best-running-shoes/) has:

• 2,381 backlinks
• 556 referring domains (423 D0follow)

Taking a look at the D0follow links.....

https://www.blackhatworld.com/attachments/runners-world-best-running-shoes-********-rd-png.115608/

For a great guide on evaluating the quality of a backlink check out https://www.gotchseo.com/backlinks/#rules which goes into more detail on the subject than I've covered briefly below.

Those are some seriously strong links. I'm not a huge fan of third-party metrics but it's obvious these sites are strong by looking at both the DR and the fact that most of these sites will be well known to most people. They even have some .edu backlinks.

So, this page has authoritative backlinks, the second important factor to examine is the relevance of those links. Let's take a look at a few of them...

A lot of the domains are general news style domains and therefore aren't 100% relevant but their power means this doesn't matter 100%. A relevant backlink would be from the running shoe manufacturers and running blogs etc. even sports or outdoors websites are semi-relevant.

There's https://searchengineland.com/relevance-vs-authority-link-value-part-1-274270 over the authority vs relevance of a backlink, however, I believe relevance beats authority. I'm not saying I'd turn down a D0follow link from somewhere like WSJ over a small blog run by a runner but if the running blog was perhaps less authoritative than WSJ then I'd take the running blog link (unless it didn't have any authority of its own and the link wouldn't pass much juice).

https://searchengineland.com/ex-googler-to-please-google-with-your-seo-forget-about-seo-136800
From what I can see many of the referring domains are pointing at an old page on the Runner's World website which has now been redirected to this new best running shoes article. So, a lot of the links don't make perfect sense in terms of context and relevance but it's clear these links still carry a lot of weight as this page ranks #1.

I'd definitely be put off by this https://backlinko.com/google-ranking-factors#backlink, it's going to be very difficult to match those number of domains and https://www.humanproofdesigns.com/offpage-seo-guide/#5 without some seriously incredible content and a big wallet. However, sometimes for these high volume terms, you don't need to rank #1 as most results on the SERP will still get a good level of traffic. In which case for a result like this, you would maybe want to evaluate the backlink profile (and other tactics I will go into below) to see whether there's a chance of ranking perhaps #2-5.

Now, if we take a look at the internal link count for Runner's World Ahrefs shows "132 groups of similar links" including links from the homepage (the homepage is usually where the majority of a websites authority and PageRank/Juice will reside).

So, Runner's World has very strong external and internal backlinks.

Sometimes you might find an article ranking for a term which you think you could improve on and "steal" backlinks from the SERP pages. You could use a host of https://www.blackhatworld.com/seo/the-complete-guide-to-link-building-tactics-20-methods.1057918/ if you choose to go for a keyword in order to gain links to your page. If the page ranking in the SERP looks like it could be improved then you could look at using the https://ahrefs.com/blog/skyscraper-technique/ in order to "steal" your competitors' backlinks.

Free Method

You can use either MozBar, Ahrefs Bar or SEO Quake in order to get an overlay of the number of referring domains and backlinks a page has.

It's worth installing all of them and searching for a few results to see which is the most accurate. Since I use the Ahrefs website I can't really comment on how accurate SEO Quake (a SEMrush tool) or MozBar is.

You're limited by free tools in that you won't be able to dive much deeper into the types of links ranking, D0follow vs Nofollow, etc. but using a toolbar is a good start and will give you a rough overview of the difficulty.

To get a rough estimate of the number of links you will need to rank you can calculate the average backlink count per page. If you have MozBar (the others also allow you to do this I believe) and Excel or Google Sheets you can do the following:

Export SERP results

You can then use the average function on Excel or Google Sheets to calculate the average number of links each page has.

Types of Websites Ranking

For this tactic, I'm talking about looking at the types of websites ranking. Sometimes it's very possible to https://www.authorityhacker.com/site-architecture/ even if you have a smaller site with fewer backlinks. In a nutshell, if you create a niche and focused website around a particular topic and cover just about everything (informative and affiliate) you can create an insanely relevant and therefore authoritative website which could outrank much bigger websites. I've personally seen micro niche websites dominate some pretty high search volume terms in the SERP and outrank some very big affiliate and general websites.

Is it a general news style website ranking (Forbes and business insider etc.) - Certainly possible to outrank for some terms

Is it a review website ranking (WireCutter, best reviews, top ten reviews, etc.) - You might have a chance of outranking if these sites are irrelevant to your keyword (perhaps your targeting best wellington boots for people with size 10 feet, I just made up this keyword btw, but all the results are general "best wellington boots")

Are there eCommerce stores ranking? Like I mentioned earlier if you see results for affiliate terms like bestseller pages etc. this is a good sign, they are ranking on authority and are not optimized for the keyword, you should be able to outrank these pages.

Are the sites in SERP thin affiliate websites - if they are and they are fairly niche and only have affiliate content on one topic (e.g. boots) it's certainly possible to outrank these websites

Are the websites authority websites in the niche - for example they are focused on a specific, usually narrow, niche and dominate the SERP with a high concentration of #1-3 spots and cover informative and affiliate content. For example, the website might be dedicated to office chairs and they might have an article on best office chairs and also how to clean a leather office chair. These websites are highgly niche relevant. A little bigger than micro niche websites these websites are trusted by Google because of there narrow focus on one particular niche.

These sorts of websites are going to be hard to outrank (even if their backlink profile doesn't appear to be very strong) and this is the type of website you should be striving to build.

The golden sites to see ranking are social media, Q&A websites, press release websites, web 2.0 (wordpress.com and Blogspot, etc.), article submission websites and forums. These will either be niche specific forums or general forums (e.g. Reddit) or sites like Quora. If you see affiliate terms with these sorts of sites you should jump for joy. They are not optimized and usually don't contain much content, this is a very good sign of low competition.


Content length/quality and on-page SEO

This is fairly straightforward. I like to skim read the page's ranking and use SEO Minion to check on-page factors such as headers, word count, meta and title, are they using appropriate referencing and linking out to authoritative sources etc.

What you're looking for is thin content, poorly written articles with mistakes, poor on-page SEO, slow page load times and an article you wouldn't consider authoritative that you would share or source/cite.

Perhaps it's a best style article but only covers 1 product or doesn't include a buyer's guide or simply misses out basic details and a FAQ section, etc.

Just like the skyscraper technique, think to yourself "could I write content that's better than this"

Sometimes a keyword won't be worth ranking for if the top few results are in incredible depth and there's nothing you can improve. This also goes hand in hand with the tactic above of examining the type of website. Big authority sites and brands will get higher organic CTR because people trust them so will click onto their page, even if it ranks a few positions down. I know I've clicked on a lower position result simply because I know and trust the website more than the other results.

A good example of this would be in the health niche. Say there's an easy keyword about a disease where the SERP doesn't have many backlinks if I saw the NHS ranking I personally wouldn't bother trying to rank the keyword, simply because I know that the NHS article would likely attract a higher CTR even if I outrank it. Note: this is more my personal opinion, you don't have to follow my advice, it's simply just my 2 cents.

Now on the other hand if the SERP is full of affiliate sites (thin style ones, not recognized brands, etc.) and the content is meh then sure I'll give the keyword a go. That's the ideal SERP, one full of thin affiliate websites. People don't have brand loyalty so they'll likely go for #1 result. If I can improve the content then I'll go after the keyword.

Moving onto on-page SEO, you want to see how https://backlinko.com/google-ranking-factors#pagelevel are for your target keyword.

An important feature of https://www.brandpoint.com/blog/search-quality-guidelines-e-a-t/ is using appropriate sources. Google wants to see you https://static.googleusercontent.com/media/www.google.com/en//insidesearch/howsearchworks/assets/searchqualityevaluatorguidelines.pdf (see p.33 section 6.7). You should do a quick E-A-T evaluation of your competitors pages to see where you could exploit areas they're lacking in. For further reading about the Medic Update and E-A-T take a look at https://exposureninja.com/blog/what-is-medic-algorithm-update/.

In terms of on-page SEO factors, you want to look at the URL, Title, Headers, Meta Description and the main content to see if they are mentioning and have https://backlinko.com/on-page-seo you'd like to rank for.

SERP History

An underlooked part of keyword research that I rarely hear anybody talk about is SERP history. I don't know of any way of doing this for free. I use Ahrefs keyword explorer for this.

I'm quoting and sourcing from https://ahrefs.com/blog/keyword-difficulty/ for this section as I can't really improve on it at all.

"A good way to learn whether Google is happy with the search results or if it’s still figuring out what’s best for users is to look at the SERP history.

Here’s SERP position history that https://ahrefs.com/keywords-explorer shows for the keyword “search engine optimization”:

The same 5 pages have been ranking at the top for quite a while, with only small shifts in their positions.

And here’s the SERP https://ahrefs.com/academy/marketing-ahrefs/lesson-3-6 for the keyword “twitter marketing”:

The pages have jumped on and off the Top10 for almost a year. And only recently, Google seems to have figured what kind of search results satisfy people the most.

In other words, SERP position history can be somewhat indicative of Google’s own level of satisfaction with their search results and thus reflect your chances of squeezing your own page onto the Top10."

Using Some Common Sense

Sometimes you come across a term which appears low competition because of the following:

• A low number of backlinks
• Small/Thin affiliate websites ranking
• Overall low domain authority (not the Moz metric but more of a general one from Google's point of view, looking at the overall backlink strength of the website, it's content and how authoritative you deem it to be)

You've used the tactics and methods I've spoken about today and you're confident you've hit the jackpot, the last step in analyzing keyword competition is using some common sense.

Firstly, https://www.blackhatworld.com/seo/keyword-research-looking-beyond-search-volume.1099983/. If the keyword is valuable and could easily be bringing in $$$+ a month then you should stop and ask yourself, is this keyword really this easy to rank? Sometimes the answer is yes and sometimes it's now, why? Because often the sites dominating in the SERP will be using PBN links. Go with your gut and have some suspicion if the competition of a keyword seems too good to be true, it's worth taking the time to analyze the SERP in further detail, perhaps by looking for hidden PBN links.

I'm not affiliated in any way with the following tool from @jamie3000 but https://www.blackhatworld.com/seo/being-outranked-but-cant-figure-out-how-probably-a-hidden-pbn-we-find-them-for-free.1090179/ is a tool you can use to try and discover hidden PBN links your potential competitors might be using.

Conclusion

I hope you enjoyed this guide and now have some further knowledge on how to evaluate keyword competition and how to analyze the SERP. Have a go at looking for some high search volume affiliate terms and some obscure niches and long tails (with lower search volume, under 1000 monthly searches) to practice.

Examples of high search volume affiliate terms:

• Best Bluetooth Headphones
• Best Running Shoes
• Best Coffee Maker

I hope you enjoyed this guide. BHW was being a little buggy and text was jumping around and paragraphs were relocating themselves everywhere so I really hope I've not missed anything out and that the formatting isn't broken as this guide took a while to write.

TLDR: How to analyze SERP/Competition

• Backlink/Referring Domain count
• Allintitle results
• Relevance
• Content length and quality
• Types of websites ranking
• SERP History

References and Further Reading

• Me
• https://ahrefs.com/blog/keyword-difficulty/
  
• https://backlinko.com/keyword-competition
  
• https://www.wordstream.com/blog/ws/2018/09/12/keyword-difficulty
  
• https://www.shivarweb.com/9822/how-to-judge-keyword-difficulty/

Click to expand...

nik2016 said:

any good method to narrow down the broad niche/ topic for find out law competition keyword?

Click to expand...