If you look at the big picture, Ahrefs is a data company.
We collect data, we process data, we store data and, finally, we build all sorts of tools that let you use our data in your marketing.
So yeah, we LOVE data! The “big” one:
Big data is a term for data sets that are so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, querying, updating and information privacy.
But before I explain what the different SEO metrics in Ahrefs mean (and how we calculate them), I'd like to brag for a bit about the monstrous infrastructure that's running behind the scenes.
Ahrefs Data Index
Here are a few core numbers, representing the size of Ahrefs’ index (you can always find them on our homepage by the way):
In our index:
- 12 trillion known links
- 3 trillion known URLs
- 200 million root domains
- 20 petabytes of storage
- 200 terabytes of RAM
- 15,000 CPU cores
Every 24 hours, our crawler visits over 6 billion web pages and updates our index every 15-30 minutes.
Creating a robot that will crawl the web and store web pages on your hard drive might sound like an easy thing to do.
But that’s until you try to achieve the crawl speed of 200 million pages per hour. Or try to store all this data in such a way that your customers can make a call to the database and have all their graphs and reports built in seconds.
That’s why the majority of our backend infrastructure was built in-house. All existing solutions simply couldn’t keep up with the volume of data we operate or, if they could, were too expensive. As of today, we’re running a custom big data database with ~85 trillion rows.
In fact, according to a recent third-party study of the "most active good bots," Ahrefs boasts the second best crawler after Google, outperforming Bing, Yahoo, Yandex, Baidu, and pretty much everyone else:
And that research was carried out before we drastically improved our crawl speed:
This week Ahrefs crawler became almost twice faster. Crawling 6 billion pages per day now, planning for 8 billion. pic.twitter.com/c4zCIxDVYV
— Dmitry Gerasimenko (@botsbreeder) February 24, 2017
So, as you can tell, we’re seriously obsessed with big data, and we absolutely love all the challenges associated with it.
It looks like our CEO isn't going to settle anytime soon, either:
Ordered 1000 servers for the first stage of a new amazing feature for Ahrefs crawler, guess what?
— Dmitry Gerasimenko (@botsbreeder) February 28, 2017
Ok, enough bragging about our (and yours too, actually) data.
Let me explain the SEO metrics that we have in Ahrefs.
"URL Rating" measures the strength of a target URL's backlink profile and the likelihood that the URL will rank high in Google.
UR is measured on a logarithmic scale from 1 to 100, with the latter being the strongest.
We often see people explain Ahrefs’ URL Rating as a replacement for Google's PageRank metric, but that's not entirely accurate. UR and PageRank aren't the same things.
We indeed started out with a PageRank-like formula, but then “UR” underwent quite a few iterations with a goal of creating a metric that would have the highest possible correlation with Google rankings.
And, as you can tell from the graph below, the URL Rating correlates with Google rankings better than any of our “unprocessed” backlink metrics:
Moz has a similar metric to our URL Rating called Page Authority, which predicts the likelihood of a page ranking high in Google. And, according to their own research study, it correlates with Google ranking slightly worse than our metric.
"Domain Rating" shows the strength of a given website's overall backlink profile.
DR is measured on a logarithmic scale from 1 to 100, with the latter being the strongest.
Domain Rating correlates with Google rankings pretty well, but not as well as the URL Rating—which suggests that you might be able to outrank high-authority sites if you build more backlinks to your page.
DR is a great metric for picking websites to build links from. As a general rule, you need to aim to get backlinks from high-DR websites because they carry more "weight."
Here are a few popular questions our customers ask about DR:
Q1: “I didn’t lose any of my backlinks. Why did my DR drop?”
A1: “That's because other sites have gained a lot of backlinks. Think of it this way: when a DR-100 website gets more backlinks, we can’t make it DR-101. So instead we push all the other websites down by 1. That's a very raw explanation of why you might see a drop in your DR while no backlinks were lost.”
Q2: “My competitor doesn't have links from high-DR sites, so why is his DR higher than mine?”
A2: “If a high-DR website links to thousands of other sites, we value their links less. But if a low-DR website only links to a few sites, we consider these links quite powerful. Your Domain Rating doesn't solely rely on the number of high-DR sites that link to you. It also takes into account how many other websites these high-DR sites link to.”
Q3: “Are links from low-DR sites bad for me? Should I disavow them?”
A3: “Low DR doesn't mean a website is bad. It just means they don't have a lot of links. If you need to identify bad links, check out this article: “An In-Depth Guide to Link Quality, Link Penalties and “Bad Links”
If you take all the websites in the world and order them by the size and quality of their backlink profile (basically by their DR), you’ll get the Ahrefs Rank.
So Ahrefs Rank #1 belongs to the website with the best backlink profile, #2 is just a bit worse, #3 is a little worse than that, etc.
You can see the full list of domains sorted by their Ahrefs Rank here.
Here's one other way to understand it.
Alexa Rank shows you how much traffic a website has relative to other websites in the world. And Ahrefs Rank shows you how good the website's backlink profile is relative to all other sites in the world.
Here’s the most popular question about Ahrefs Rank:
Q: “My backlink profile is growing, but my Ahrefs Rank is going down. Why?”
A: “Because other websites are getting new backlinks faster than you. What happens here is very similar to my above answer about a drop in your DR.”
As you’re browsing backlink reports in Ahrefs’ Site Explorer tool, you might notice a “Live/Fresh” switch that changes the numbers in your reports when you toggle it.
What is the difference between the two?
Like I mentioned above, Ahrefs' backlink index is updated every 15-30 minutes with all the new backlinks our crawler has found.
But not only do we crawl new pages, we also re-crawl the old ones. And, therefore, some of the links that were seen live might disappear by the time we re-crawl them.
We remove all these “dead links” from our “Live” index right away, but they stay in the “Fresh” index for 3-4 months so you have enough time to act on this information.
The “Fresh” index shows that Brian Dean used to have a backlink from an article at Outbrain, but it got removed for some reason:
Knowing that, Brian could reach out to Outrbrain and persuade them to put it back.
A link saved is a link earned, right?
"Keyword Difficulty" shows how hard it would be to rank in the top10 Google search results for a given keyword.
KD is measured on a logarithmic scale from 1 to 100, with the latter being the hardest.
This SEO metric will save you a lot of time doing keyword research. But it should only be used for pre-vetting keywords, as nothing beats good old manual review of the SERPs.
Please also note that our KD scale is non-linear. So KD 50 is not a "medium" keyword difficulty:
In fact, I wrote a pretty detailed article on the concept of "keyword difficulty" in general and how our own metric fits there: "How To Gauge Keyword Difficulty And Find The Easiest Keywords To Rank For".
And you can read about the rest of the SEO metrics in the Keywords Explorer tool here: "Keywords Explorer 2.0 by Ahrefs: Setting new standards for keyword research".
Here are some of the most popular questions about Keyword Difficulty:
Q1: “Do you take "on page" factors into account when calculating KD?”
A1: “No. Ahrefs' Keyword Difficulty is based solely on the backlink factors of the top10 ranking pages.
We don't look at the keyword usage in a Title/URL/H1 because our recent study of on-page SEO factors revealed it has only a minor impact on your ability to rank.
Calculating the overall relevancy of a page might be a good addition to KD, but we haven't tackled it just yet.”
Q2: “The #1 result is very strong! And your KD is way off!”
A2: “Ahrefs' Keyword Difficulty score only shows you how hard it would be to rank in the top10, not #1. When you're in the top10, a lot of other ranking factors kick in, so it's almost impossible to accurately calculate your chances of ranking #1.”
Q3: “How accurate is your KD score compared to other tools?”
A3: “It's impossible to objectively compare KD scores from different tools because there's no "base KD value" to compare against. Each SEO professional uses his "gut feeling" as a base value, which obviously differs from person to person.
And yet, the only KD study that was carried out to this date showed that Ahrefs has the most accurate KD score of all available solutions.”
Search volume was one of the hottest topics in SEO last year.
Ever since that happened, our customers keep asking us this question:
"How accurate are the Search volume numbers in Ahrefs?"
Google Keyword Planner is known to have quite a few "dirty secrets," which causes SEO professionals to question the accuracy of their data.
So you'll be glad to know we no longer rely on GKP alone for calculating our Search volumes.
We pair their numbers with the data from clickstream sources, allowing us to calculate more accurate search volume numbers.
I've shared more details about it here.
And I plan to write a dedicated article about the issue with Google's search volume and how we solve it. So if you don't want to miss it, please subscribe to the Ahrefs Blog.
For any website you put into Site Explorer, we'll show you the number of keywords it ranks for and the estimated amount of organic search traffic it gets:
And if you switch to the "Organic search" tab of the "Overview" report, you'll see this beautiful graph:
So how accurate are these numbers and how do we calculate them?
The database of keywords in Site Explorer is approaching 200 millions for the United States alone.
But, obviously, even a database that big can't cover all possible search queries that people might put into Google. That's why we tend to underestimate the total number of keywords a website is ranking for.
This is our estimation of how much traffic a target website or URL gets from organic search. We calculate it based on the keywords we see that website or URL ranking for.
And because we don't have all possible keywords (see above), we tend to drastically underestimate the search traffic of a given website.
For example, for the Ahrefs Blog, we underestimate organic search traffic by 5x:
But as soon as you try to compare two websites from the same niche, you get a very accurate picture.
Ahrefs shows that our company blog gets around 10x more search traffic than my personal blog:
Which is exactly the case, if you look at the numbers from Google Analytics:
In other words, the organic traffic numbers you see in Ahrefs should not be treated as exact values but, rather, should be used for comparing websites with each other and seeing trends in your search traffic over time.
Back To You
So these are the metrics we get asked about the most.
I hope that this article helps you better understand Ahrefs' reports and make them more actionable.
If you have any other questions about the numbers you see in Ahrefs, feel free to ask them in the comments and I’ll be happy to help you.