1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Why “Keyword Difficulty Calculators” Suck + How To Make Your Own Calculations [2,087 Words]

Discussion in 'White Hat SEO' started by Jared255, May 1, 2017.

  1. Jared255

    Jared255 Jr. Executive VIP Jr. VIP Premium Member

    Joined:
    May 10, 2009
    Messages:
    2,005
    Likes Received:
    1,911
    Location:
    Boston, MA
    I see noobs everywhere trusting keyword difficulty scores blindly without understanding how they are calculated or what they mean. The takeaway from this post is that you should NOT trust keyword difficulty scores because they are terribly inaccurate for the most part. Blindly trusting keyword difficulty scores is a great way to pick a crappy keyword and spend tons of time and money trying to rank, then failing.

    If you wish to understand why keyword difficulty scores suck - and what the better method of “calculating difficulty” is - continue reading.

    How Keyword Difficulty Is Calculated on Any Major Keyword Tool

    Any keyword difficulty score is going to take a hodgepodge of metrics from each of the the top 10 results, apply them to a formula, get a “difficulty score” for each result, then average them together and spit out a number between 1 and 100.

    Metrics like these will be consulted:

    • InTitle / InURL / InDescription - Does the keyword appear in the title, URL, or meta description of the result?
    • Moz / Majestic ranks - How do Moz and/or Majestic, two large SEO firms, calculate the difficulty of this result? (Moz uses page authority and domain authority, and Majestic uses Trust Flow and Citation Flow).
    • Backlink count - How many backlinks does this result have?
    • Social count - How many social shares does this result have?

    The formula will assign an arbitrary value to each of these determinants and give a difficulty score for each top 10 result. Then, they are averaged together, and that is your “keyword difficulty score”. Each major keyword tool uses a slightly different formula to calculate difficulty but they all operate by considering the same metrics such as InTitle / backlink count / etc.

    Why This Method of “Calculation” Is Terribly Inaccurate

    For each and every metric that is consulted in the makeup of the “keyword difficulty score”, there are a variety of ways that the automated formula can misread the SERP result and display something inaccurate.

    InTitle / InURL / InDescription

    Google will recognize similar words and phrases as being the exact same - it’s a semantic search engine. A result may have slightly different wording from the keyword you are checking, which will result in a NO for the keyword difficulty calculator, but a YES in the eyes of Google.

    Moz / Majestic ranks

    While these scores are usually somewhat accurate, they are not perfect, nor are they a reflection of what Google believes to be the strength of the page. For example, it’s very easy to manipulate DA and PA with low-quality links. Google will still consider the backlink profile to be WEAK, but because of the sheer number of links, Moz will consider the backlink profile to be STRONG.

    Backlink count

    This is the big one. There is not a single keyword tool that is able to delve into the quality of the backlinks that a result has - it’s an enormous task and probably close to impossible. So instead, they put weight on the number of backlinks, and not the quality of the backlinks. A page with 10 strong editorial links will be shown as WEAK to a keyword difficulty calculator, but a page with 5,000 forum profile links will be shown as STRONG to a keyword difficulty calculator, and WEAK to Google. (Link count in general tells you very little about backlink profile strength.)

    Social count

    Sometimes, platforms will factor in metrics such as number of social signals. No one really knows how powerful metrics like social signals are. So if a result has 1,000 Facebook likes, a keyword difficulty calculator might see that as STRONG, whereas Google might NOT CARE. At the same time, a result with no social activity might be seen as WEAK by a keyword difficulty calculator, and Google might NOT CARE.

    When you combine all of these possible missteps, there is a LOT of room for error, and in general, keyword scores are riddled with errors. In general, the keyword difficulty calculator will misread at least one of the determinants above, which will lead to a higher or lower keyword difficulty score than the true difficulty. Add in a few misreads and it’s totally out of whack.

    This means you will sometimes perceive the keyword as too difficult and not pursue it (even if it is easy). And you will sometimes perceive it as easy when it is difficult, which means you will waste time and money trying to rank something impossible. Keyword calculations cannot be blindly trusted because they can lead you down a path which wastes your time and money.

    When Keyword Difficulty Scores Can Be Useful

    Keyword difficulty scores can tell you when a keyword is completely impossible or unbelievably easy. That is the only real way you should be using keyword difficulty scores: to know at a glance if a keyword is super hard or super easy or in the middle. But there are still ways that a tough result can read as low competition and an easy result can read as high competition… so don’t trust it completely, ever! There is too much on the line - your time and money - to be lazy and pick a bad keyword.

    Keyword difficulty scores are in general useless for judging anywhere from low- to medium-competition keywords - aside from telling you that they are in that range. Two keywords with the exact same competition can easily return 20 or 40. 20 will be seen as “very easy” by the tool, and 40 will be seen as “medium” by the tool. In fact two different variations of the same keyword might spit out vastly different numbers because of the missteps in the keyword calculation algorithm.

    There is too much room for error to get any accurate gauge on in-the-middle keywords - which is what most of us on BHW are after. All it can tell you is that the keyword is probably in the middle somewhere, or that it’s extremely weak or extremely difficult.

    How To Create Your Own “Keyword Difficulty Calculator”

    Intuition and knowing how to judge SERP results is the key. It’s a skill and a VERY USEFUL one at that. I can’t install the skill in your brain, but I can tell you how to get started analyzing SERP results and getting better at it yourself.

    Common sense

    Did you just search “dog food” and see a bunch of static category pages for e-commerce sites like Petco and Amazon with thousands of “dog food” products within each category? If it’s something like that, it’s probably not worth looking at. All blatantly obvious keywords are probably not rankable.

    Search volume

    Are you looking at a 70,000 USA p/m search term with $1+ CPC? It’s probably not easy. The more profitable a keyword, the more competition it will have. In general, it is not feasible for the average BHW member to target high-volume, very-profitable keywords. While high-volume and low-competition keywords do exist, you probably won’t find them as a beginner. Use search volume as a guide - do not look at any base keyword above 10,000 if you are a beginner is my advice.

    Looking at top 10 keyword inclusion

    Is the keyword genuinely missing from the top 10 results, or is it there in a different variation or a different way of saying it? Chances are, in 2017, Google is not returning results that are completely irrelevant, so make sure that you aren’t leading yourself on. But, it is worth looking at something like this: if you are looking at “blackhat SEO”, but all of the top 10 results just talk about “SEO” and do not mention “blackhat” or any variation or different way of phrasing it, you may be onto something.

    Looking at DA/PA

    Are any of these numbers close to 100 or 0? If so the score may not be entirely accurate. For example, a Weebly page will return a very high DA, but Weebly results aren’t particularly powerful because anyone can make a page on Weebly. All parasites will have high DA but that doesn’t matter as much because it’s a parasite. At the same time, if a result has 1 PA, or 3 PA, Moz might just not have enough data on that result to calculate a PA - it’s probably not actually 1.

    Looking at backlinks

    All of the above is important but backlink profile is by far THE most important. THIS IS THE BIG ONE THAT ALMOST ALL KEYWORD DIFFICULTY CALCULATORS PERCEIVE INCORRECTLY.

    Backlink count doesn’t really mean sh*t. You have to look at the actual backlinks and see what their value is. Pop a result into a backlink checker (Moz, Ahrefs, Monitor Backlinks, Backlink Watc, etc.) and take note of what you see.

    Are they spam links? PBN links? Honest-to-God editorial links? A lot of spam links won’t mean much. Just a few strong PBNs or editorial links can mean a lot. You need to understand what types of links have power. Once you understand that, you can analyze backlink profiles of potential keywords to see if you can match or exceed the power of the profile. If you can, you might have a winning keyword.

    But it’s not ALL about backlinks - for example, a static category page on an authority site might not have a ton of backlinks, but each of the posts or pages within that category might, which will feed the category page in power. So even if the category page has no backlinks, it still might have a lot of link juice powering it.

    As another example, a highly-relevant page on HuffPo will still have a lot of power, even if it genuinely has 0 links. Part of making a decision on competitiveness is balancing backlink power with internal power.

    At the end of the day you never know for sure if a keyword is easy - you don’t have access to Google’s exact algorithm - but with enough trial and error you will be able to immediately spot a good one.

    Everything else

    Take it into account, but don’t put too much weight on it. If a result has thousands of social shares… those might be purchased, or worthless to Google - again, no one knows. That’s just one example but there are others.

    Steps to Analyzing a Keyword

    This is how I do it, rather than trusting inaccurate keyword tools.

    • Determine if the keyword is remotely feasible. Use common sense, general keyword scores, and the factors discussed in this guide to make that determination.
    • Delve into backlink profile strength. If you determine the backlinks of the results are nothing special, continue on.
    • Weigh backlink strength against internal link strength. If there are authoritative but low-link results at the top, could you beat the results with a new low-authority site, but a very strong link profile?
    • If you think you can… go for it! One of two things will happen. You will either have found a good keyword and rank and make money, or you won’t, and you will learn a valuable lesson on keyword analysis.
    • Be careful and check for mistakes. Don’t get absorbed in the excitement of potentially finding a keyword and make a decision based on incomplete information. Really look at the keyword’s top results to check for anything out of place. Did you overlook anything? It’s best to continue doing keyword research within a niche for a few days before you decide on anything if you are new. Rushing into a bad keyword will only waste your time and money.

    Overall, keyword research is a wide-ranging skill that takes determination to get good at. You can’t get specific keyword advice, because most people will just steal your keyword if it’s any good. You have to do it yourself. Always keep your findings secret and only discuss general things.

    Go now and start trying to find a keyword. Use this guide as a starting point. Once you do some research, you will outgrow this guide, and all of your new ideas and tactics will be yours to own and find lucrative keywords with. All it takes is brute force until you find one good keyword, then building off patterns you recognize in the viable keyword.

    Questions? Post below! I’ll reply to everyone.
     
    • Thanks Thanks x 43
  2. ZuppaSoup

    ZuppaSoup Newbie

    Joined:
    Mar 13, 2017
    Messages:
    47
    Likes Received:
    4
    Gender:
    Male
    This is an important one. Sometimes when I'm targeting keyword with lower monthly search, I accidentally can get traffic from keyword with huge monthly search on that article, even if that phrase of keyword not in the article at all or very minimum.
     
  3. validseo

    validseo Jr. VIP Jr. VIP Premium Member

    Joined:
    Jul 17, 2013
    Messages:
    917
    Likes Received:
    527
    Occupation:
    Professional SEO
    Location:
    Seattle, Wa
    I agree with a lot of what you just said. :) I have found a lot of success in obtaining the actual measurement data. I am very suspicious of 3rd party scores that claim to represent what Google is looking for.

    upload_2017-5-1_10-37-24.png
     
    • Thanks Thanks x 4
  4. radiant13

    radiant13 Power Member

    Joined:
    May 19, 2010
    Messages:
    611
    Likes Received:
    360
    Man, you're on a roll today. Will have to come back later to read this one in it's entirety.
     
  5. MisterF

    MisterF Jr. VIP Jr. VIP

    Joined:
    Nov 29, 2009
    Messages:
    7,800
    Likes Received:
    6,296
    Occupation:
    Conference Organiser, Business Advisor.,
    Location:
    JADIP
    Home Page:
    Really informative post, once again. Thanks.
     
  6. Skyebug77

    Skyebug77 Jr. VIP Jr. VIP

    Joined:
    Mar 22, 2012
    Messages:
    2,185
    Likes Received:
    1,584
    Occupation:
    Marketing
    Location:
    Portland,Or
    This is really well put together piece on this topic. Im waiting on your next post.
     
  7. Breezly

    Breezly Newbie

    Joined:
    May 1, 2017
    Messages:
    48
    Likes Received:
    15
    Gender:
    Male
    I enjoyed reading it Jared. Would you agree under 1,000,000 search results typically means the niche is easier? Also factor in not many news sites ranking in the top of search engines.
     
  8. seogirl19

    seogirl19 Newbie

    Joined:
    Jan 31, 2016
    Messages:
    22
    Likes Received:
    8
    Gender:
    Female
    Location:
    On my feet
    wow !!! it's very informative deep read, to much depends on 3rd party tools
     
  9. s7ryker

    s7ryker Junior Member

    Joined:
    Dec 26, 2014
    Messages:
    166
    Likes Received:
    49
    Gender:
    Male
    Occupation:
    Whatever Works
    Location:
    On my Way
    I was going to start a new thread but since i saw yours I'm gona to post here
    I'm new to keyword research and i notice that tolls give a lot of false results and i found a keyword which in my humble opinion is a no competition keyword but i just want to see what you think about it.
    [​IMG] [​IMG] [​IMG] [​IMG]
    Nobody is targeting the keyword..there is no optimization. DA10 PA1 on first page , Pinterest and although Keyword revealer says 25 i think no competition keyword here also allintitle search is without quotes, I'm sorry if i disturb your thread.
     
  10. donsiemen

    donsiemen Regular Member

    Joined:
    Mar 26, 2012
    Messages:
    304
    Likes Received:
    81
    Occupation:
    I.T tech
    Location:
    Maryland
    Very informative
    Thanks Jared
     
  11. issorc

    issorc Jr. VIP Jr. VIP

    Joined:
    Feb 20, 2016
    Messages:
    712
    Likes Received:
    575
    Gender:
    Male
    Good information for beginners in keyword research in here @Jared255
    Some of these points are really important if you're diving into keyword research for the first time, especially that MOZ is not that accurate most of the time.
     
  12. Jared255

    Jared255 Jr. Executive VIP Jr. VIP Premium Member

    Joined:
    May 10, 2009
    Messages:
    2,005
    Likes Received:
    1,911
    Location:
    Boston, MA
    Interesting, I usually only get more specific traffic, e.g. if my site is about "blue toasters" maybe I get traffic for "teal 4 compartment toaster" but not usually anything upwards in generality e.g. getting traffic from "toaster". Kudos to you, love that unexpected traffic, so delicious.

    Impressive stuff.

    Thanks man, and just wait, I have a lot more coming. :)

    You got it, thanks for the kind words, really appreciate it.

    Thanks again Skyebug, and just wait, have about a dozen more posts like this one coming.

    Thanks for the kind words Breezly. I don't think under 1,000,000 indicates something is easy but I do think that anything over 1,000,000 is probably off the table for 99.99% of BHW members.

    Thanks for the kind words! You need to use third-party tools to your advantage but not trust them completely.

    No problem. Hard to tell without knowing the specifics. I personally don't use allintitle or any of the search modifiers to determine difficulty. In your case here's what I'd do. Looks like your keyword is "best X for X", and the "for X" isn't covered. Confirm that. Look at page source for all of top 10 results. See if that word or anything similar is mentioned. You HAVE to make sure there are no synonyms that you are missing. If it's not mentioned, you might be able to beat those results for the "for X" term in particular. But if it is included, or any keywords are that Google considers similar, I'd stay away.

    Again, I don't know the specifics, so take the above advice with a grain of salt.

    Thanks for the kind words donsiemen, glad you liked.

    Thanks issorc. Yeah, Moz can be good, but it can be bad. Overall I really like Moz for looking at pages on smaller sites and/or homepages for smaller sites, but for any decent-sized site I've found it to be somewhat misleading a lot of the time.
     
    • Thanks Thanks x 2
    Last edited: May 1, 2017
  13. villecity

    villecity Junior Member

    Joined:
    Jan 5, 2013
    Messages:
    153
    Likes Received:
    14
    Thanks for providing us with this informative information!
     
  14. umerjutt00

    umerjutt00 Jr. VIP Jr. VIP

    Joined:
    Oct 28, 2011
    Messages:
    3,908
    Likes Received:
    2,168
    Occupation:
    Ninja
  15. Setox

    Setox Power Member

    Joined:
    Apr 30, 2015
    Messages:
    581
    Likes Received:
    243
    Occupation:
    CPA Hunting
    Location:
    MALTA MAYBE ..
    For a SEO newbie like me who just started doing Kw research the wrong way ! this is really a Goldmine thread & Go-to thread . Thanks .
     
  16. stevesolo

    stevesolo Jr. VIP Jr. VIP

    Joined:
    Jul 3, 2014
    Messages:
    899
    Likes Received:
    242
    Gender:
    Female
    Occupation:
    Content Marketing
    Home Page:
    Really awesome man. It's a nice startup guide for newbies and helped me a little also. Thanks for the share:)
     
  17. Vladamir

    Vladamir Power Member

    Joined:
    Jan 4, 2012
    Messages:
    561
    Likes Received:
    335
    Gender:
    Male
    Occupation:
    Restore Old Boats
    Location:
    Zihuatanejo, Ixtapa, Mexico
    Jesus , do you fill out that whole excel chart yourself or have a va do it.

    I use 3 tools, keyword revealer longtail pro and majestic to check backlinks of top 3 when those first two tools give me a similar score. I have had good luck with those....
     
  18. vancleve

    vancleve Jr. VIP Jr. VIP

    Joined:
    Apr 4, 2014
    Messages:
    120
    Likes Received:
    20
    Finally, a comprehensive post about checking for keyword difficulty. This should be of immense help for both beginners and experienced SEOs, as keyword research can make or break an entire campaign. Taking the time to uncover all pertinent details proves much more worth it over simply working on a keyword that a certain tool deems easy to rank for. Good job, mate!
     
  19. kullaravi

    kullaravi Junior Member

    Joined:
    May 1, 2011
    Messages:
    102
    Likes Received:
    23
    very useful read. This should be the first thing to be read by anyone who gets in to keyword search and then only they should get trying the tools.I loved the 'common sense' approach in making sense out of the 'keyword difficulty' .
     
  20. validseo

    validseo Jr. VIP Jr. VIP Premium Member

    Joined:
    Jul 17, 2013
    Messages:
    917
    Likes Received:
    527
    Occupation:
    Professional SEO
    Location:
    Seattle, Wa
    Nope. It is all done by software