AhrefsBot
Learn about the web crawler that powers Ahrefs.
| Version | 7.0 |
|---|---|
| Bot type | Good (Identifies itself, has an official moniker) |
| Kategorie | Marketing |
| Robots.txt befolgen. | Ja |
| Obeys crawl delay | Ja |
| User-Agent String | Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) |
| Reverse-DNS-Suffix | ahrefs.com |
| IP-Adressbereich |
Website status
Was ist der AhrefsBot?
AhrefsBot ist ein Web-Crawler, der die Datenbank sowohl für Ahrefs, ein Online-Datentoolset, als auch Yep, eine umsatzbeteiligte Web-Suchmaschine, speist. Er ist der zweitaktivste Crawler nach Google und besucht alle 24 Stunden über 8 Milliarden Webseiten und aktualisiert seinen Index alle 15–30 Minuten.
Our bot indexes fresh, accurate information about websites and their content, and how they link to each other. This data is incredibly useful and can be harnessed in many ways:
Search with Yep to find top-quality content and all the information you need, all while supporting an alternative search engine that aims to put money in the pockets of your favorite content creators.
If you’re a website owner, use Ahrefs to analyze your website's structure, content, and backlinks to provide insights into its performance – then optimize it to improve user experience, rank higher in search engines, and get more traffic.
When it comes to web-scale research, Ahrefs lets you dig into how the popularity of a website or specific topic has changed over time, which websites are linking to what types of content, how many times a brand has been mentioned recently, and so on.
Rank | Bot | Owner | Category |
|---|---|---|---|
1 | GoogleBot | Search Engine Crawler | |
2 | AhrefsBot | Ahrefs | Search Engine Optimization |
3 | Qualys | Qualys | Security |
4 | BingBot | Microsoft | Search Engine Crawler |
5 | Amazonbot | Amazon | AI Search |
6 | Google Image Proxy | Page Preview | |
7 | Google AdsBot | Advertising & Marketing | |
8 | YandexBot | Yandex | Search Engine Crawler |
9 | Google API | Webhooks | |
10 | Synthetic Bot | DataDog | Monitoring & Analytics |
Bots, sortiert nach Anzahl der Anfragen. Daten bereitgestellt ab Dezember 2024
Questions? We have answers
Beachtet AhrefsBot die robots.txt?
Ja. Absolut.Wir folgen den Disallow und Allow Regeln in der robots.txt.How do I control AhrefsBot on my website?
Since AhrefsBot strictly follows the robots.txt file on your website, you're in full control of what it does.To change the frequency at which AhrefsBot visits your site, just specify the minimum acceptable delay between two consecutive requests in your robots.txt file:User-Agent: AhrefsBotCrawl-Verzögerung: [value](Wobei die Crawl-Verzögerung in Sekunden angegeben wird.)If you want to stop AhrefsBot from visiting your site, add these two lines to the robots.txt file on your server:User-Agent: AhrefsBotDisallow: /Bitte beachte, dass AhrefsBot etwas Zeit benötigt, um die Änderungen in deiner robots.txt-Datei zu erkennen. Dies geschieht vor dem nächsten geplanten Crawl.Also, if your robots.txt contains errors, AhrefsBot won’t be able to recognize your commands and it'll continue crawling your website the way it did before.Read more about robots.txt at www.robotstxt.org.If you think that AhrefsBot is misbehaving on your website or if you have any questions, please reach out to [email protected] and we'll do our best to help.
Du hast andere Fragen?
Falls du weitere Fragen hast,
kontaktiere uns, damit wir dir helfen können.