The hard part is figuring out what it takes to rank #1 for each keyword and use that information to prioritise your list and plan your SEO strategy.
Many keyword tools (Ahrefs included) try to solve that problem by showing you a “keyword difficulty” or “keyword competitiveness” metric – but can you rely on their judgement?
Well, the goal of this article is to give you the definitive answer to this question.
No one really knows how Google ranks pages
Basically, the entire SEO industry is nothing but hundreds of thousands of people using trial and error to figure out how Google ranks pages.
We also know that Google is experimenting a lot with machine learning and artificial intelligence algorithms, which should completely revolutionize “search” in the next few years.
So where am I going with this?
If you want to determine keyword difficulty with 100% accuracy, you need to use exactly the same algorithms that Google uses to rank pages.
So does any third-party tool have access to Google’s ranking algorithms?
Could they develop information-processing algorithms that could boast the same level of sophistication as Google?
That’s why no keyword difficulty checker is perfect and each tool can only give you their best estimate.
But even an estimate is better than nothing, right? And besides, certain tools are much more accurate than others (wink).
Important: a lot of people who are new to SEO mistakenly rely on the “Competition” metric that they see in Google Keyword Tool. Please be advised that this metric has nothing to do with ranking difficulty and only shows how many advertisers are bidding to show their ads in the search results for a given keyword.
How to determine the ranking difficulty of a keyword
The only way to learn how difficult it would be to rank on top of Google for a specific keyword is by carefully analysing the pages that already rank there.
Ideally, you’d want to vet these pages for all of Google’s 200+ ranking factors. But since no one (except Google) really knows how much each individual factor contributes to the resulting ranking of a page, it makes sense to focus on the biggest ones: links and content.
Let’s use one of the keywords we’re targeting in the Ahrefs Blog as an example: “anchor text”
The quickest way to see the number of backlinks the Top10 ranking pages for this keyword have is to put it into Ahrefs’ Keywords Explorer tool and scroll down to the “SERP overview” report:
The “Domains” column shows how many unique websites link to a given page. And it’s not that hard to see a general pattern: the more sites link to a page, the higher it ranks in Google.
In fact, we’ve studied the correlation between a page’s number of referring domains to its position in Google across 2 million keywords, and it turned out to be a strong ranking factor:
One other interesting takeaway from the above graph is that the number of referring domains to a page has a better correlation with Google rankings than just a raw number of backlinks. So, as a general rule, it’s better to get one link from 10 different websites than 10 links from a single website.
But apart from the sheer quantity, there’s also a quality factor in place: a small number of high-quality links may trump a larger number of lower quality ones.
For that we have a metric called URL Rating (or “UR”).
You can see from the graph above that UR correlates with Google ranking much better than the raw number of linking domains. That’s because Ahrefs’ URL Rating takes into account the quality of backlinks (to a certain extent) and was specifically designed to reflect the ability of a page to rank well in Google (read more about Ahrefs’ metrics here).
And yet, even with UR (which is the highest correlated metric in the SEO industry) we’re only scratching the surface of how Google would process backlink factors.
There’s just too much to consider:
- Where is the link is located on the page?
- Is that link likely to be noticed and/or send traffic?
- What is the anchor text of that link?
- What is the surrounding text of that link?
- How many other backlinks are on the page?
- At what pace is the page acquiring new backlinks?
- & so on.
Authority of a domain
A lot of SEOs believe that the so-called “domain authority” (or “domain rating”) has a big influence on a page’s ability to rank.
But at the same time many SEO professionals are convinced that such a thing as “domain authority” does not exist.
So who’s right and who’s wrong?
Well, here at Ahrefs, we’ve studied the correlation of domain-level backlink factors across 2 million keyword searches and plotted them alongside some key page-level factors:
As you can tell from our data above, domain-level factors have significantly smaller correlation with rankings than page-level factors. And yet that correlation is still quite solid.
Does this mean that Domain Rating helps you rank higher?
I’m afraid we can’t confirm that based only on this correlation. Correlation ≠ causation.
But what our data suggests is that you should be able outrank high-DR websites if you have more links coming to your page.
And this wraps up my very brief overview of how to approach keyword difficulty from a backlinks standpoint.
Usually SEOs won’t go too deep in reviewing a given SERP: they will just look at the number of linking domains and UR/DR of the top-ranking pages and settle with that information. But for some important keywords you may want to go as far as reviewing the actual backlinks, where they come from and what would it take to replicate them.
It is true that you can easily outrank pages with vastly more backlinks if they’re lacking relevance to the search query.
Here’s a keyword that perfectly illustrates what I mean: “chocolate lab”
Looks like the pages with only 6–20 referring domains are outranking the pages with 900‑1000 referring domains.
How is that possible?
Well, if you open that Wikipedia page with over a thousand referring domains, you’ll see that “chocolate Labrador” is only a small sub-section of a very big article:
Meanwhile, the articles ranking above that Wikipedia page are entirely dedicated to this specific breed:
This is a perfect illustration of how relevant content can outrank even the strongest backlink profile.
But don’t get too excited about it just yet.
What we see in this example is called “lack of relevant content.” The top-ranking results are targeting a broader search query (Labrador retriever), rather than a very specific one that people are searching for (chocolate lab).
That is a massive opportunity for relevant content to shine, and that’s how those two articles got to the top without a lot of backlinks.
But you don’t see this kind of thing very often.
Usually what you get in the SERP is “slightly imperfect content” (at best). The top-ranking results are 100% relevant to a search query, but they could do a slightly better job of giving visitors what they’re looking for.
This kind of SERP won’t give you the same level of competitive advantage as “lack of relevant content,” where you can rank without backlinks.
So how do you know if the search results for your keyword are lacking relevant content, giving you a good chance to beat them without links?
And how do you make your own page 100% relevant to a given keyword in the eyes of Google?
Let me try to address these two things.
“Conventional on-page SEO” vs topical relevance
Imagine you put your target keyword in Google and see that the top-ranking pages don’t use that keyword in their Title/URL/Headline.
This indicates that you can easily outrank them if you just use the keyword in your page’s Title/URL/Headline, right?
The best practices of on-page SEO in 2017 are not as straightforward as they were back in 2010.
Back then, Google didn’t have fancy things like Hummingbird and RankBrain, so it needed some very strong clues to understand what your page was about. Putting your exact-match keyword in the Title/URL/Headline of your page gave a strong competitive edge over the pages that weren’t doing that.
But this trick doesn’t work anymore. Today, Google is smart enough to understand what your page is about even when a target keyword is never mentioned on the page.
In fact, by studying 2 Million keyword searches we have discovered that almost 75% of the pages that rank in Google’s Top10 don’t have a single mention of an exact-match keyword in their content.
Check out the SERP for the keyword “guest writing” to see what I’m talking about:
Clearly, Google understands that things like “guest writing,” “guest blogging” and “guest posting” are closely related. So if you perfectly optimize your page for the keyword “guest writing” in accordance with these old-school on-page SEO best practices, that won’t give you any competitive edge at all.
How to make your page relevant
Or should I re-phrase that as “how to make your page MORE relevant than the pages that currently rank in the Top10, so Google will rank your page higher even with fewer backlinks”?
Well, I’m afraid there’s no easy and straightforward way to do it.
In order to make your page perfectly relevant to Google, you first need to understand how Google interprets search queries and matches them to topics and entities that it extracts from web pages.
Sounds complicated, right? That’s because it is.
And why should they?
Since Google is getting so smart that it almost “reads” the pages of your website, why should you even bother adjusting your pages to meet some complex criteria of its algorithms and not just “write for humans”?
Well, the important word here is “almost.” Despite its impressive complexity, Google is still a machine, and if you understand how it works and can adjust your pages accordingly, you’ll be one step ahead of everyone else.
We’re going to properly cover the topic of “new on-page SEO” in one of our upcoming articles, so now I’ll leave you with 2 quick tips:
- Use Ahrefs’ Site Explorer tool to analyse top-ranking pages for your target keyword and see what other keywords they also rank for. This will give you some clues as to what topics Google thinks they’re relevant to.
- Open the Top10 pages that rank for your target keyword and use one very sophisticated tool to extract topics from them – your brain. If you Google around and study all the pages related to your topic, you’ll naturally build a good thesaurus of words and topics that will help Google identify your own page as perfectly relevant.
In most cases “relevance” and “user intent” go hand in hand. But sometimes a perfectly relevant search result might not give the user what he’s looking for.
In this case, Google will always favour “user intent” over “relevance.”
Sounds confusing? I have a great example for you.
If you search for “online survey” from the United States, you get search results that look like this:
Nine search results suggest tools for creating online surveys and 1 search result offers work-from-home “online survey jobs.”
But what happens when we search for exactly the same keyword from the United Kingdom?
This time only 5 of the search results offer tools for online surveys, while the other 5 offer “online survey jobs.”
This example shows that people might be looking for different things when they search for a general keyword that may have multiple meanings.
And Google has somehow identified that most people in the US are looking for an online survey tool, while a lot of people in the UK are also interested in making some money by participating in online surveys.
But how does Google know what people are looking for?
It hasn’t been officially confirmed, but the rumours are they might be looking at things like:
- how long people stay on the page after clicking on it in the search results (a metric known as dwell time);
- whether people click on any other search results or just settle with the one they picked first;
- whether people get what they were looking for from their first search or if they will keep refining it and clicking on more search results.
And these kinds of things can sometimes outweigh the topical relevancy of a page.
I mean, if more people in the UK started clicking on search results related to “online survey jobs,” Google would see that and drop a bunch of “online survey tools” results from the front page – even if they were perfectly relevant to the keyword “online survey” and had tons of backlinks.
A good way to learn whether Google is happy with the search results or if it’s still figuring out what’s best for users is to look at the SERP history.
Here’s SERP position history that Ahrefs’ Keywords Explorer tool shows for the keyword “search engine optimization”:
The same 5 pages have been ranking at the top for quite a while, with only small shifts in their positions.
And here’s the SERP ranking history for the keyword “twitter marketing”:
The pages have jumped on and off the Top10 for almost a year. And only recently, Google seems to have figured what kind of search results satisfy people the most.
In other words, SERP position history can be somewhat indicative of Google’s own level of satisfaction with their search results and thus reflect your chances of squeezing your own page onto the Top10.
And that winds up our discussion of the factors you should review when assessing a keyword’s difficulty.
Now let’s discuss Ahrefs’ Keyword Difficulty metric and how it can make things easier for you.
Ahrefs’ Keyword Difficulty
Once you put your keyword into Ahrefs’ Keywords Explorer tool, the first metric you’ll see will be the Keyword Difficulty score:
We measure KD on a scale from 0 to 100, with the latter being the hardest to rank for.
But that doesn’t really help you understand what KD 15 or KD 65 means, right?
I figured it would be best to use a Q&A format to explain Ahrefs’ Keyword Difficulty score and how to use it:
1. What does Ahrefs’ KD score mean?
Our keyword difficulty metric gives you an estimate of how hard it would be to rank in the Top10 search results for a given keyword.
Attention: I didn’t say, “Rank #1 for a given keyword.” I said, “Rank in Top10 for a given keyword.”
This little nuance causes a lot of confusion: People see an insanely strong page ranking #1 and decide that our KD score is not accurate. But if you look at pages #2–10, they may be a lot weaker and thus easier to outrank.
Here at Ahrefs, we believe Google relies heavily on backlinks to identify the pages that deserve to rank in the Top10.
But once your page reaches the front page of Google, all sorts of other important factors kick in:
- which search result attracts the most clicks
- which page shows the best behavioural factors
- do people keep looking for more results by refining their search query?
And lets not forget about those sophisticated topic modeling algorithms that I mentioned earlier.
In other words, creating a keyword difficulty score that accurately predicts the #1 is as easy as building our own Google here at Ahrefs.
That’s why for now we’ve settled with predicting the chances of ranking in the Top10, which we do quite accurately.
2. How do you calculate KD?
We look at how many backlinks the Top10 ranking pages for a given keyword have.
We don’t take into account things like: domain rating, the age of the website, usage of the keyword in the Title/URL/H1, etc (I’ll explain why below).
We also don’t differentiate between dofollow and nofollow links, because the the SEO community still haven’t decided if nofollow backlinks help you rank or not.
3. Why don’t you take into account any “on-page SEO” factors?
First of all, let’s define the “on-page SEO factors” we’re talking about.
Most likely, you’re referring to things like:
- Using exact match keyword in the Title/URL/Headline;
- Keyword density of your target keyword in content;
- Length of content;
- Number of social shares;
- Outgoing links to authority sites;
- Page load time;
Well, numerous studies (ours included) have confirmed that these kinds of things have a minuscule correlation with rankings compared to backlink factors.
So even if we were to include them in our KD calculation, they would change the resulting KD score by no more than +/- 3 points. That’s not significant enough to bother spending CPU resources on it.
But if we’re talking about the advanced on-page SEO concepts I’ve mentioned above – topic modeling, TF-IDF, entity salience, etc. – we’re obviously working in this direction. We’re just not ready yet to apply what we have to our KD formula.
4. How accurate is your Keyword Difficulty score?
Given that Ahrefs boasts the world’s best database of live backlinks, our Keyword Difficulty score represents the most accurate picture of how competitive a SERP is backlink-wise.
The accuracy of our metric was confirmed by a third-party test of existing keyword difficulty tools, where Ahrefs won first prize:
But even though Ahrefs is more accurate than any other tool, we don’t recommend that you blindly base your SEO decisions on our KD score alone.
It can be a great “first filter” to weed out the keywords that would require way too many backlinks to rank, but then you’ll have to look at the actual SERP and vet the top-ranking pages manually in the way I explained above.
5. How do I read your KD scale?
Because Ahrefs’ Keyword Difficulty metric is tied to backlinks and nothing else, the scale is pretty straightforward.
Each KD score translates into an average number of referring domains that Top10 ranking results have:
KD 0 = 0 Ref. Domains
KD 10 = 10 Ref. Domains
KD 20 = 22 Ref. Domains
KD 30 = 36 Ref. Domains
KD 40 = 56 Ref. Domains
KD 50 = 84 Ref. Domains
KD 60 = 129 Ref. Domains
KD 70 = 202 Ref. Domains
KD 80 = 353 Ref. Domains
KD 90 = 756 Ref. Domains
From the above graph you can see that our KD scale is exponential, so there’s a vastly bigger difference in referring domains between KD 80 and KD 90 than between KD 20 and KD 30.
6. What range of KD can be considered “easy”?
This question is tricky, because what seems to be “easy” for one person might turn out to be “insanely hard” for someone else.
And yet, based on conversations with many of our customers, we figured that a universally accepted KD scale should look like this:
Even for us at Ahrefs, getting ~40 websites to link to our page is pretty hard to do.
7. What KD range can I safely target with my website?
You can easily identify your “safe” KD range by looking at the keywords you currently rank for.
Put your website into Ahrefs’ Site Explorer tool, go to the “Organic Keywords” report and apply “Position” and “Volume” filters to find your best keywords that rank in the Top5:
In the screenshot above, I filtered all keywords with Volume over 500 searches per month where ahrefs.com ranks in Positions 1–5.
That resulted in 26 keywords (many of which are duplicated because they rank in SERP features).
Then I exported that report and copy/pasted these 26 keywords into Keywords Explorer.
It automatically removed the duplicates and showed me aggregated data for the 16 keywords that were left:
As you can see from the “Difficulty distribution” graph, most of the keywords in my list fall into the KD 40–60 bucket.
This means that we can safely target any keywords with KD up to 60.
Replicate the same steps for your own website and you’ll see the maximum possible KD that you can target.
8. Why are a lot of KDs in my report gray?
Here’s a screenshot of what it looks like:
We have 2 types of Keyword Difficulty in Keywords Explorer:
- Cached — we regularly process SERPs for millions of keywords and calculate the KD score based on the pages we see ranking there at this time. Some keywords are updated every day, while others might take up to a month to renew. We show cached KD in gray to illustrate that it’s not up to date;
- Fresh — once you click the “Get metrics” button next to any keyword, we’ll get the fresh SERP and calculate a KD based on the pages that rank there at this very moment. The resulting number will be assigned a color in accordance with our scale above.
Cached KD might not be very accurate, because SERPs are subject to change, but it can be extremely handy when you need to filter a huge list with a couple thousand keyword ideas to just a few that don’t require many backlinks to rank for:
Time to find some easy keywords to rank for!
So now you’re armed with all the knowledge you need to accurately gauge the ranking difficulty of a keyword.
Ahrefs KD score is a good starting point for your research, but please make sure you always check the SERP manually before making a decision to put your time and money into some keyword.
See you on the front page of Google 😉