New in September: Traffic value, JS crawling, “delayed” links & more

Tim Soulo
Tim is the guy responsible for marketing and product strategy at Ahrefs. But most importantly he’s the chief evangelist of the company. Learn more

Article stats


    I had to skip reports for July and August, because the only two things we were working on were the release of new Rank Tracker and the upcoming release of Site Audit tool. 

    But we made quite a few cool updates in September:

    1. Ahrefs now crawls links in JS

    This is a pretty big deal actually, so it was already announced with a dedicated article: 

    Ahrefs crawlers are now rendering web pages and executing JavaScript

    Right now we only execute JS on pages that have over 15 referring domains pointing at them. 

    But that is 30 Million pages every 24 hours!

    That said, we’re the only SEO tool that can crawl JS at scale as of today. Which we’re insanely proud of.

    2. Ahrefs’ Big Data in numbers

    We often say that Ahrefs tools are powered by Big Data, but we rarely give you updates on what these numbers are.

    So we allocated a bit of time last month to create a cool page that would list all the most important numbers behind Ahrefs:

    Ahrefs in numbers

    For the time being, it is just a snapshot of our data (and infrastructure) as of September. 

    But because we continuously improve our service and all these numbers grow consistently, we’ll probably develop an API to pull the most recent data every time you load that page.

    3. “Traffic value” metric

    Every marketer knows that traffic by itself is a vanity metric (in most cases).

    So we’ve added a monetary value right next to the amount of organic search traffic that Ahrefs estimates for any website or URL:

    Traffic value” basically shows how much you would have to pay using AdWords to get the same amount of visits from the same search queries that this website ranks for organically (learn more).

    Some of the search queries have very high Cost Per Click, because a lot of people want to place their ads in the search results. But other search queries are of no interest to any advertisers, so the CPC is practically 0.

    This is why you might see websites with a small amount of traffic that costs a lot, as well as websites with huge traffic that is dirt cheap.

    4. “Delayed” new backlinks

    In Site Explorer tool, under “Backlinks > New” report, you may have seen a new tag: 

    Delayed

    To explain what it means, I will have to tell you a little bit about how Ahrefs crawler works:

    Let’s say we have crawled 1 page and discovered links to 10 other pages on it. 

    We won’t go and crawl all these 10 pages right away, because each of them will probably have another 10 links, which we’ll have to also immediately crawl. And each of those will have another 10, and so on…

    So what we do instead is save all the URLs that we have found on that page to our “smart scheduler.”

    This “smart scheduler” then uses a bunch of factors to determine which of these URLs have to be crawled first and which ones can be crawled later.

    Sidenote.
    One of the main factors that will influence our decision to crawl an unknown URL is (obviously) how often we find links to that page from other pages that we crawl. 

    But even though we didn’t crawl a certain URL right away, we have recorded the date when we first discovered it. 

    And this is how this Delayed tag was born.

    The whole purpose of “New Backlinks” report in Site Explorer is to show you the most recent backlinks. 

    But it may happen that you get a link from a very low quality page that our “smart scheduler” will not feed to our crawler for quite a while. So by the time we crawl this page and discover your backlink, it won’t really be “new.”

    This is why we decided to add a Delayed tag to such backlinks. 

    It indicates that we have discovered the linking page more than 90 days ago, but only crawled it just now and discovered that it links to you.

    5. More data on “Top Pages”

    One of my favourite reports in Site Explorer tool is “Organic search > Top pages.” 

    It helps me to quickly assess which pages bring the most traffic to the websites of my competitors.

    But that report was just a bit incomplete. So we’ve added two more columns to it:

    • The first column shows how much search traffic each page gets compared to the total search traffic of that website.
    • And the second shows how many other websites link to each page.

    Both numbers are insanely valuable. Now you can see the value of each page relative to other pages of that website and you can also understand how many backlinks you’ll have to get in order to replicate that result on your own website.

    Sidenote.
    The number of referring domains (RD) that you see in this report is a cached number. Ahrefs stores organic traffic data and backlink data in two separate indexes and there’s a big technical challenge to pull data from both of them simultaneously. So we had to cache RD values for this report. 

    6. Improvements in Content Gap

    Content Gap tool shows you what keywords your competitors are ranking for in search that your own website doesn’t rank for.

    But what it didn’t show was the actual URLs of the pages, that ranked for these keywords.

    Which we have fixed!

    By clicking on website’s ranking position for each search query, you’ll go straight to the ranking page.

    But not only that! 

    You can also click on a small caret to the right from this number and you’ll see a popup that has quick links to other Ahrefs reports for that URL as well as some of its core metrics:

    That’s it!

    We’re hoping that you will enjoy these new updates to Ahrefs tools that were made in September.

    And if you have any questions or feedback, please do not hesitate to let us know about it!

    Tim Soulo
    Tim is the guy responsible for marketing and product strategy at Ahrefs. But most importantly he’s the chief evangelist of the company. Learn more

    Article stats

    Shows how many different websites are linking to this piece of content. As a general rule, the more websites link to you, the higher you rank in Google.

    Shows estimated monthly search traffic to this article according to Ahrefs data. The actual search traffic (as reported in Google Analytics) is usually 3-5 times bigger.

    Get notified of new articles

    46,903 marketers are already subscribed to Ahrefs blog. Leave your email to get our weekly newsletter.

    • Harris Roberts

      Nice job guys! Thanks for the update 🙂

    • Aleem Iqbal

      Thanks for the Update

    • Ajay Lyca

      Nice updates to an already amazing tool. When can we expect Site Audit to be released?

    • MD Alomgir Skyb

      Wow Good

    • ahrefs site içi optimizasyon, içinde çözüm üretme harika olacak. Diğer sorunlar bazı bağlantıları çok geç tespit ediyor. Kısa sürede çözüm bulacaktır.

    • Brittany Elizabeth

      Wow great update. Thanks the Ahrefs Team.

    • Great job. Thanks

    • Excellent progress all around. I know the JS is huge but haven’t tried it yet, excited to test it out.

      The biggest new thing for me is the addition of the additional two columns in Top Pages, as that’s also my favorite report.

    • Links in JS! love this one.

    • buen trabajo! como siempre estais haciendo una labor muy grande comparado con otras herramientas seo del mercado! poca competencia os supera! felicidades!

    • Very cool. Cheers !

    • Alexandra Dinu

      i am a seo blonde, this is huge for me but thanks anyway

    • Great updates.

      Regarding the “Traffic value metric”. Can you please include in the pop-up info note that it’s for Monthly traffic? It will require any user to click on “Learn more” first to figure if it’s for daily or monthly traffic. Just a suggestion.

    • Like always, awesome!

    • This is a small but valuable article!

    • I like the part improving top page.
      It’s really helpful 😀