{"id":172884,"date":"2024-03-13T12:14:56","date_gmt":"2024-03-13T17:14:56","guid":{"rendered":"https:\/\/ahrefs.com\/blog\/?p=172884"},"modified":"2025-06-23T18:18:33","modified_gmt":"2025-06-23T23:18:33","slug":"link-index-comparison","status":"publish","type":"post","link":"https:\/\/ahrefs.com\/blog\/link-index-comparison\/","title":{"rendered":"You Can\u2019t Compare Backlink Counts in SEO Tools: Here\u2019s Why"},"content":{"rendered":"\n
Google knows about 300T pages on the web. It\u2019s doubtful they crawl all of those, and at least according to some documents from their antitrust trial we learned they only indexed 400B. That\u2019s around .133% of the pages they know about, roughly 1 out of every 752 pages.<\/div>\n\n\n\n

For Ahrefs, we choose to store about 340B pages in our index as of December 2023.<\/p>\n\n\n\n

At a certain point, the quality of the web becomes bad. There are lots of spam and junk pages that just add noise to the data without adding any value to the index.<\/p>\n\n\n\n

Large parts of the web are also duplicate content, ~60% according to Google\u2019s Gary Illyes<\/a>. Most of this is technical duplication caused by different systems. However, if you don\u2019t account for this duplication, it can waste more resources and create more noise in the data.<\/p>\n\n\n\n

When building an index of the web, companies have to make many choices around crawling, parsing, and indexing data. While there\u2019s going to be a lot of overlap between indexes, there\u2019s also going to be some differences depending on each company\u2019s decisions.<\/p>\n\n\n\n

Comparing link indexes is hard because of all the different choices the various tools have made. I try my best to make some comparisons more fair, but even for a few sites I\u2019m telling you that I don\u2019t want to put in all of the work needed to make an accurate comparison, much less do it for an entire study. You\u2019ll see why I say this later when you read what it would take to compare the data accurately.<\/p>\n\n\n\n

However, I did run some tests on a sample of sites and I\u2019ll show you how to check the data yourself. I also pulled some fairly large 3rd party data samples for some additional validation.<\/p>\n\n\n\n

Let\u2019s dive in.<\/p>\n\n\n

Contents<\/div><\/span><\/span><\/span><\/a><\/div>\n\n\n\n