We’re powered by seriously big data

Ahrefs collects, processes, and stores large amounts of data for search marketing professionals.

Similar to search engines, we crawl the entire web 24/7 and store petabytes (1PB = 1000TB) of information about live websites – like how they link to each other and what keywords they rank for in search results.

We're also really proud of our backend infrastructure. Since no existing solution could keep up with the volume of data we operate, most of our software infrastructure was built in-house.

4000
Servers
695.1
K
CPU cores
4
PB
RAM
32
PB
HDD
505
PB
SSD

$900
M
The amount we’d have to spend on the cloud in 3 years without our own infrastructure.
Watch reel

Meet Yep1, our superfast supercomputer

Yep1, Ahrefs’ supercomputer, ranks among the Top 50 fastest supercomputers worldwide by Flops (floating point operations per second).

Data supplied as of May, 2024. See full report

23.32
PFlops

Most active web crawler in the SEO industry

AhrefsBot is the most active web crawler in the “Search Engine Optimization” category on Cloudflare Radar. It’s also the 2nd most active web crawler in the world.

Every minute, we crawl
5
M
Pages
There are
170
T
Rows in our key-value database
/

Rank

Bot

Owner

Category

1

GoogleBot

Google

Search Engine Crawler

2

AhrefsBot

Ahrefs

Search Engine Optimization

3

Qualys

Qualys

Security

4

BingBot

Microsoft

Search Engine Crawler

5

Amazonbot

Amazon

AI Search

Bots ranked by number of requests. Data supplied as of December, 2024


Content index

Our content index powers Content Explorer, which is basically a content search engine.

Every 24 hours, we discover
1
M
New pages
Every 24 hours, we update metrics for
1
M
Pages

Total content index size

17.5B

Pages


Keyword index

This powers Keywords Explorer, our keyword research tool for finding what your prospective customers are searching for online.

217
Locations

Total keyword index size

28.7B

Filtered keywords

110B

Keywords ever seen

2.4BUnited States
1.5BBrazil
1.3BUnited Kingdom
1.3BIndia
1.1BGermany
1.1BFrance
1.0BCanada
845MRussian Federation
819MSpain
807MIndonesia
774MItaly
773MJapan
761MMexico
726MAustralia
714MVietnam
697MPhilippines
670MPoland
668MTurkey
631MNetherlands
613MArgentina
575MSouth Korea
548MMalaysia
548MUkraine
515MThailand
508MColombia
462MPakistan
460MTaiwan
434MSingapore
423MSouth Africa
422MEgypt
411MSweden
401MUnited Arab Emirates
398MBelgium
392MSwitzerland
367MBangladesh
360MSaudi Arabia
358MPeru
353MChile
340MIreland
339MRomania
328MIran
314MPortugal
313MCzech Republic
312MAustria
308MMorocco
293MAlgeria
289MHong Kong
285MGreece

2.4BUnited States
1.5BBrazil
1.3BUnited Kingdom
1.3BIndia
1.1BGermany
1.1BFrance
1.0BCanada
845MRussian Federation
819MSpain
807MIndonesia
774MItaly
773MJapan
761MMexico
726MAustralia
714MVietnam
697MPhilippines
670MPoland
668MTurkey
631MNetherlands
613MArgentina
575MSouth Korea
548MMalaysia
548MUkraine
515MThailand
508MColombia
462MPakistan
460MTaiwan
434MSingapore
423MSouth Africa
422MEgypt
411MSweden
401MUnited Arab Emirates

2.4BUnited States
1.5BBrazil
1.3BUnited Kingdom
1.3BIndia
1.1BGermany
1.1BFrance
1.0BCanada
845MRussian Federation
819MSpain
807MIndonesia
774MItaly
773MJapan
761MMexico
726MAustralia
714MVietnam
697MPhilippines


Organic search traffic index

Our database of search queries and the amount of search traffic they send to top-ranking pages. It powers search traffic estimations in Site Explorer.

208
Locations
1
M
Keywords
153MUnited States
31MIndia
23MUnited Kingdom
23MFrance
23MJapan
21MBrazil
21MGermany
19MCanada
18MSpain
17MRussian Federation
16MAustralia
15MItaly
13MMexico
12MNetherlands
12MIndonesia
11MTurkey
9.3MPoland
6.7MColombia
6.6MPhilippines
6.4MArgentina
5.9MThailand
5.6MVietnam
4.6MUkraine
4.6MSweden
3.5MTaiwan
3.4MMalaysia
3.2MChile
3.0MSouth Africa
3.0MDenmark
2.9MNorway
2.9MPeru
2.7MSaudi Arabia
2.7MPakistan
2.7MAustria
2.7MEgypt
2.5MCzech Republic
2.5MNew Zealand
2.3MSouth Korea
2.3MBelgium
2.2MUnited Arab Emirates
2.1MPortugal
2.0MRomania
1.9MFinland
1.9MHungary
1.9MSingapore
1.9MSwitzerland
1.9MGreece
1.7MHong Kong

153MUnited States
31MIndia
23MUnited Kingdom
23MFrance
23MJapan
21MBrazil
21MGermany
19MCanada
18MSpain
17MRussian Federation
16MAustralia
15MItaly
13MMexico
12MNetherlands
12MIndonesia
11MTurkey
9.3MPoland
6.7MColombia
6.6MPhilippines
6.4MArgentina
5.9MThailand
5.6MVietnam
4.6MUkraine
4.6MSweden
3.5MTaiwan
3.4MMalaysia
3.2MChile
3.0MSouth Africa
3.0MDenmark
2.9MNorway
2.9MPeru
2.7MSaudi Arabia

153MUnited States
31MIndia
23MUnited Kingdom
23MFrance
23MJapan
21MBrazil
21MGermany
19MCanada
18MSpain
17MRussian Federation
16MAustralia
15MItaly
13MMexico
12MNetherlands
12MIndonesia
11MTurkey


Backlink index

The world's largest index of live backlinks, updated with fresh data every 15–30 minutes. This index powers backlink data in Site Explorer, and is used to calculate our proprietary metrics: Ahrefs Rank, Domain Rating and URL Rating.

External backlinks history

35.0T

Records

Pages in index
493.9
B
Domains (post-vetting)
500.4
M
Internal backlinks
28.7
T