Keyword Search Volume: Things you didn’t know you don’t know

Tim Soulo

Tim is the guy responsible for marketing and product strategy at Ahrefs. But most importantly he’s the chief evangelist of the company. Learn more about Tim

Article stats

  • Referring domains 41
Data from Content Explorer tool.
    Search volume is one of the most important and probably the most frequently used metric in SEO. Entire businesses are being built out of the popularity of certain search queries in Google.

    But as more and more keyword research tools enter the market, users start noticing that different tools report different search volumes for the same keywords.

    And, most importantly, these search volumes are often different from what Google Keyword Planner is showing (which most people regard as the only accurate source of keyword data).

    How accurate is the keyword search volume in Ahrefs?”

    Our support team gets this question almost every other day. So I decided to address it here in all possible detail.

    Where do keyword research tools get their search volume data?

    It’s no secret that the vast majority of keyword research tools get their data from Google Keyword Planner (directly, or via the services that scrape this data and re-sell it).

    So, if everyone is using GKP as their source, how come different tools report different search volumes for the same keywords?

    And why, in most cases, does it not match what Google Keyword Planner shows?

    Sidenote.
    I didn’t include search volume data from Ahrefs in this comparison table, because our search volume is actually designed to be different from GKP. More on this later in this post.

    WARNING! Please do not use this small comparison to identify the tool that was the closest to GKP. A test of six keywords says nothing about accuracy. I’d have to test at least 100k keywords to make the comparison fair. So this small experiment is just to illustrate that no tool will match GKP data in 100% of cases.

    The search volume that you see in Google Keyword Planner is a rounded annual average, which means they re-calculate it every month.

    Let’s look at the search trend for the query “independent films”:

    As you can see, it’s been losing its popularity lately, so the annual average in Google Keyword Planner is getting a bit smaller each month.

    This means that all keyword tools that use GKP as their source have to update the search volumes of all their keywords once a month in order to provide you with the most “accurate” data.

    Which is quite a challenge if you have a few million (or billion) keywords in your database, because Google does not have an API for pulling search volume.

    That’s why no tool in the above comparison table reported the right search volume for all 6 keywords — they just don’t update their database frequently enough.

    How accurate is the search volume in AdWords?

    Most users would be perfectly happy if their keyword tool of choice would always display the same numbers that they see in Google AdWords.

    But professional SEOs have always been questioning the data that they get from Google: “Google Keyword Planner’s Dirty Secrets.”

    Here’s the official definition of the search volume in GKP:

    “The average number of searches for this keyword and its close variants based on the targeting settings and date range you’ve selected. You can use this information to see how popular your keywords are during a certain time of the year.”

    So let’s dive a bit deeper and see what’s behind that definition.

    Annual average

    Annual average” is a rather fun thing on its own.

    I mean look at the keyword “Christmas”, that has a search volume of 550,000 according to Google.

    If you pull up the annual trend for that search query, you’ll see that it barely gets 100k searches per month for most of the year, then peaks at over 3 million in December:

    So how useful is that “annual average” here?

    Christmas” is of course an extreme case, but you get the point. Most of the search queries don’t have a flat search volume trend all year long, so the “annual average” might be way different from next month’s value.

    Buckets

    Let’s go back to the annual trend for the search query “independent films”:

    Half of these bars report the search volume of 2,400 and the other half says 2,900.

    You don’t have to be Grigori Perelman to calculate the average of 2,650 across 12 months of data.

    But GKP search volume for “independent films” is actually 2,400, which is off by 250 searches.

    Well, that’s because the annual average that GKP shows is rounded into “buckets.”

    Which makes an already inaccurate number even more inaccurate.

    Close variants

    Guess what happens if I put these 3 search queries into GKP:

    • SEO
    • search engine optimization
    • search engine optimisation

    They will be combined into a single keyword:

    So Google won’t show me if there’s any difference in search popularity between these three searches. Which makes a big difference if you want to rank organically, because the search results for these three queries differ quite a bit.

    But for AdWords advertisers the difference in organic search results doesn’t matter, since their ads are always on top. So Google Keyword Planner makes things useful for advertisers, but not SEOs.

    More search volume “clues” in Google’s tools.

    The “rounded annual average” that you see in Google Keyword Planner is actually not the only way that you can gauge the search popularity of a keyword.

    They also have this tool called Performance forecasts.

    To get there, you need to add a few search queries to your plan, and then click the “Review plan” button:

    From there, by following a simple process described here, you’ll be able to get the “impressions” data for your list of keywords. Even for the close variants that GKP was previously hiding from you:

    I tested quite a few keywords in this forecaster tool, and the number of impressions was always different from the original search volume. Which is kind of fun.

    But that fun doesn’t end there.

    There’s one more source of search volume data in Google, but it only works for the keywords that your website is already ranking for.

    I’m talking about the Google Search Console.

    My article was consistently ranking in Google top3 for the “keyword research” search query, so I can use GSC to calculate the number of impressions it has generated in the past 28 days:

    I got 19,766 impressions for that keyword in the last 28 days (from all countries). But GKP reports the search volume of 22,200 searches per month for that keyword, which is quite higher.

    So I took 5 keywords that ahrefs.com consistently ranks for in top5 search results and compared the data from three Google sources (in the United States):

    • Google Keyword Planner;
    • Performance forecasts (in Google Keyword Planner);
    • Google Search Console.

    As you can probably tell, there’s not much consistency between these three sources.

    And there’s also no way to tell which of these sources is the most accurate.

    Clickstream — a new “source” of keyword data.

    Common sense tells us that Google is the only possible source of keyword data, because they are the only ones who know how many times people around the world search for different things.

    Is there any other way to learn what people search for in Google?

    Yes and no.

    YES, because some applications that you install on your computer and plugins that you add to your browser request your permission to collect certain data from you.

    This is called clickstream.

    Then these apps and plugins sell their clickstream data to bigger companies. These bigger companies aggregate data from as many sources as they can in order to re-sell that big pile to companies like Ahrefs, which in turn do all sorts of cool things with it.

    NO, because the number of people in the world who have “clickstream apps” on their computers is relatively small.

    Sidenote.
    In case you got worried that someone (besides Google, Apple & Microsoft) might be tracking you — relax. All legit applications only collect anonymised clickstream data. And then they mix your data with my data, so they know what both of us searched for in Google, but they don’t know who exactly searched what.

    So, all in all, raw clickstream can’t be considered as a reliable source of data, even if you find all re-sellers and buy all clickstream data that is available.

    But clickstream data is super useful when you cross-reference it with data from Google Keyword Planner.

    Which is exactly what we do here at Ahrefs.

    And here’s why it is so cool:

    Monthly updates

    Thanks to clickstream, we can easily update our search volumes every single month and catch all the trends:

    By the way, right now we calculate our search volume as 30-months average, but will most likely switch that to 12-months average soon.

    Sidenote.
    The search volume numbers in Ahrefs are also rounded off a bit, but that is just to make the numbers look prettier. We don’t use any “buckets” that would distort our search volume data.

    Un-grouping close variants

    We can also un-group the close variants that Google Keyword Planner is grouping together:

    Search volume ≠ Clicks

    The fact that people search for something doesn’t mean that they will click on any of the search results.

    As you can see, a lot of people were searching for “Donald Trump age” during the US elections, but they weren’t clicking on the search results, because Google has an instant answer to this query:

    But there’s more!

    Take a look at this screenshot:

    How come we report nearly the same traffic from #1 position in Google for two keywords, while the search volume of these keywords differs by over 5x?

    Well, we use clickstream data to determine the CTR curve for every individual keyword (where we have enough data to be certain).

    So for the keyword “moz” we see that ~55% of clicks go to #1 result. Which is expected, since it’s a branded search query.

    And for the keyword “SEO” we see that only ~10% of clicks will go to #1 result, because most searchers won’t actually click on anything when searching for it:

    That is pretty cool, right?

    But there’s one last thing that you need to know about the search volume.

    WARNING: Search volume is a bad predictor of search traffic!

    This probably sounds super counterintuitive, but I have a great example to illustrate my point.

    Let’s compare the search volume of these two keywords:

    SEO tips” has almost twice the search volume of “submit website to search engines.” Which kind of suggests that if you rank your article for that keyword, you’re going to get twice more traffic.

    Right?

    WRONG!

    Let’s take the top ranking pages for both search queries and plug them into Ahrefs to see how much search traffic they get in total:

    Looks like the page that ranks for “SEO tips” gets 10x less traffic than the page that ranks for “submit website to search engines,” while the former has a 2x higher search volume.

    That’s because a web page will rarely rank in Google for a single search query alone.

    In fact, we’ve recently studied that an average ranking page will also rank for about 1000 longer-tail keywords.

    And here’s what is happening in this particular case.

    SEO tips” is a rather specific search query and there are not too many alternative ways to put it:

    • Seo tips — 2,700
    • seo optimization tips — 500
    • seo tips and tricks — 300
    • google seo tip — 100
    • easy seo tips — 30
    • tips for seo — 20

    So the total search volume of this group of search queries is mostly condensed at “SEO tips” — which is clearly the most comprehensive way to put it.

    And the overall number of these relevant search queries is rather limited, that’s why the top ranking page for “SEO tips” is only ranking for 148 keywords in total.

    Submit website to search engines” is very different in this sense. There are a lot of alternative ways to phrase this request, all of which sound quite plausible:

    • submit website to search engines — 1,300
    • submit site to search engines — 500
    • submit url to search engines — 500
    • submit to search engines — 300
    • submitting site to search engines — 300
    • add website to search engines — 250
    • submitting to search engines — 200
    • submitting page to google — 200
    • submitting my website to google — 200
    • submitting your website to google — 200
    • submit urls to search engines — 200
    • & many-many more

    This is how the top-ranking page for “submit website to search engines” ranks in Google for almost 4,000 other search queries, all of which bring a ton of search traffic.

    And you would have never discovered this amazing traffic opportunity if you’re only looking at the search volume and not analysing the top-ranking pages.

    Sidenote.
    Here’s another cool example of search queries where traffic doesn’t match the search volume: “cool wallpapers” and “how to check your credit score”.

    If you’re a user of Ahrefs, we have a cool SERP Checker tool within Keywords Explorer, where we pull top10 ranking pages for your keyword and show you how much traffic they get in total (along with many other useful metrics):

    I know it is tempting to assume that search volume of a keyword and total traffic of the top ranking pages are highly correlated, but we’re now running a study that proves otherwise:

    But I’m going to publish the results of this study as a separate post, once we finish it and double-check our numbers.

    Final words

    Google gets better and better each year at understanding all sorts of peculiar search queries that people put into it.

    This means that “the long tail” of search is getting even longer and the search volume of an individual keyword is becoming more and more obsolete.

    So if you’re not already building your SEO strategy around “topics” instead of “keywords,” it is about time to adopt this new mindset.

    Tim Soulo

    Tim is the guy responsible for marketing and product strategy at Ahrefs. But most importantly he’s the chief evangelist of the company. Learn more about Tim

    Article stats

    • Referring domains 41
    Data from Content Explorer tool.

    Shows how many different websites are linking to this piece of content. As a general rule, the more websites link to you, the higher you rank in Google.

    Shows estimated monthly search traffic to this article according to Ahrefs data. The actual search traffic (as reported in Google Analytics) is usually 3-5 times bigger.

    Get notified of new articles

    46,390 marketers are already subscribed to Ahrefs blog. Leave your email to get our weekly newsletter.

    • Very well then! It’s time to sit back and think about what I know and may not really know.
      Thank Tim, you just light me up 😀

    • You have perfectly shown the process of keyword researching.

    • Jonathan Jones

      Always loved your method of calculating search volumes, very innovative indeed.

      Would have liked for you to have shown us the SV for ‘ahrefs’ to compare it to your GSS data. 🙂

      And, any plans to break down the volumes by device? i.e. 50% of traffic is mobile, 40% desktop and 10% tablet

    • Umar Arshad

      very well written informative article as all ..keep it up

    • Very helpful article Tim — thanks for sharing it 🙂

    • It would be great if the monthly and yearly volumes of the keywords are shown.

    • Philip DiPatrizio

      You weren’t filtering US in your example for impressions of [keyword research]. Maybe it’s closer to 5400 then? Probably not, but figured it was worth pointing out.

      • Please re-read that part again.. everything seems correct.. I first checked “global” impressions for “keyword research” in GSC.. and then, within the comparison table, I used “US only” numbers

    • You just made it a lot easier to explain Google Search Console impressions vs. organic traffic to our clients now.

      Cheers!

    • Very useful article !! thanks for sharing it … Keep it up ..

    • Great article Tim! appreciate the explanations and new insight.

    • Thanks! Hope it will help you going forward with your keyword research, Lisa 🙂

    • I really enjoyed reading this article Tim! You definitely shed some light on questions I did not know the answers to, great informative read.

    • Very nice article!! Look forward to reading your next study about the correlation(not) between search volume and traffic.

    • Thank you for this article. I have always compared search volume data from different tools and the discrepancies are sometimes confusing. This clarifies things for me.

    • May I copy your neural connections that exist in your brain? You write incredible articles on incredibile topics! A big fan of you!

    • Angga

      Awesome post Tim! Can you more elaborate about this “building your SEO strategy around “topics” instead of “keywords,”
      Thanks!

    • Thank you for this article

    • Impressive information

    • https://uploads.disquscdn.com/images/18ec72da14a63d313815eaa200f1303746eae07caadd5dba1293a742665e1e7f.png

      Keyword Research” has 22,200 Avg Monthly Searches Globally and 5400 in USA.

      I think, 19,766 Impressions for the query “keyword research” are global.

      • Yes, my mistake. Fixed this part, thanks!

        The numbers in my comparison table that goes below are for US only and perfectly correct 😉