"Wait ... what just happened to my damn site, and why is it nowhere to be seen on the first page of Google?!"
... Sometimes diagnosing a loss of SEO visibility is rather straightforward, unfortunately.
In most cases, however, provided you're not entirely out of luck, things are a bit more subtle.
I mean, even if you do lose some SEO visibility, it doesn't happen across the board, but rather affects specific areas of your site, here and there, so to speak.
So how to keep your finger on the pulse? How to stay updated with what's really going on with your site SEO-wise, and most importantly, what to do in case you discover any shenanigans?
Well, this is where the tools we're listing today come into play. Come dig in.
Note: The original idea for the method that I'm just about to share comes from Asaad Dookhy - the SEO Lead at Decathlon UK - who was generous enough to share his strategy with us and let us describe it in this post.
What you'll learn:
- How to diagnose a loss in SEO visibility,
- How to identify the areas of your site that got affected the most,
- How to notice any correlations between known Google algorithm updates and your visibility loss,
- How to evaluate if a loss of backlinks is to blame,
- How to assess if there was an industry-wide shakeup,
- Which of your competitors benefited the most and how,
- What to do to fix it.
1. Google Analytics (GA), for getting a good top-level view
Okay, let's get the obvious out of the way. You were perfectly aware that GA is going to appear on this list somewhere.
1.1 Finding out if there's anything to worry about
GA is simply invaluable when diagnosing any sort of SEO trouble. Quite naturally, if your site loses any of its SEO visibility, this is going to be reflected in your overall organic traffic stats. Examining those numbers gives you a general overview of the situation and its seriousness.
1.2 Identifying individual areas of the site that got hit
Next, in case anything is going on, go to the Site Content section. There, you're going to identify the areas of your site that got hit the most. Remember, we're just looking into the organic traffic, not the overall numbers.
Starting from your most popular content, make your way down the list (as far as you want to go) and single out the specific pages that lost the highest numbers compared to their general trends from a couple of weeks back when everything was fine.
Also, it's a good idea to go back a couple of years to see if the drop isn't a seasonal thing (often the case in some niches).
(Asaad actually advises using Searchmetrics for this part of the job, but considering that the most usable version of the tool is $2,000 a month, we're having hard time including it in this list. Hence using GA instead.)
1.3 Checking what's going on on mobile
One more thing we're going to do with GA is check how the site is performing on mobile. With the mobile-friendly algorithm update now live, it's possible that some of your visibility loss (or even all of it) comes from mobile.
This can be looked into by going to Audience > Mobile. There, select the checkbox next to mobile, and then plot the new row:
Is there anything shaky going on, or is the mobile graph just a scaled down version of the main sessions graph?
2. Google Search Console (GSC), for confirmation
(Formerly known as Google Webmaster Tools)
GSC has a lot of great uses for a site owner and especially if you want to assess your SEO losses in certain areas.
2.1 Checking the site's mobile-friendliness
First off, since we were just talking about mobile a minute ago, a good place to start is to check what GSC has to say about the mobile-friendliness of your site.
Just go here and input the URLs from your site that were the most likely to get hit. Those can be the URLs that you discovered earlier through GA (those that lost traffic) or, if you're running your site on WordPress, example URLs representing different types of pages that you use (for instance: one blog post page, one category page, the homepage, etc.).
This will be enough in most cases as you can generally assume that if, say, one of your blog posts isn't mobile-friendly then they probably all aren't, and you just need to adjust that naughty single post template.
2.2 Any manual actions taken against your site?
GSC will also let you know about any manual actions that might have been taken against your site.
This can be viewed if you go to Search Traffic > Manual Actions. If everything is in order, you'll see this:
(Screenshot courtesy of Search Engine Land. Sorry, I'm lucky enough to not have any such screenshots of my own, so I had to borrow one.)
2.3 Checking your search engine impressions
[Impressions represent] the number of times pages from your site appeared in search results.
In short, what this means is that any serious problem with your SEO visibility will be reflected in your impressions graph. You can see it by going to Search Traffic > Search Queries:
If there's a significant drop, you're in trouble.
2.4 Looking into your brand terms
One final check you can perform in GSC is looking into your brand terms and how they perform. In other words, what you're checking here is whether the loss of SEO visibility could be caused by the brand itself losing popularity.
This you can do by going back to Search Traffic > Search Queries and filtering the list to show only search terms including your brand name. Again, pay attention to the trends and see if you've experienced any sudden drop in impressions.
3. SEMrush, for identifying the troublesome keywords
Okay, so at this point you know that your site got hit and what areas received the most damage. Now let's look at the other side of the equation - finding the keywords to blame.
3.1 Checking where you rank for specific keywords
Doing this in GA can be somewhat difficult as there's a lot of guessing work involved. After all, Google is openly against any sort of keyword-based rank tracking.
Luckily, SEMrush still seems to function just alright. In it, go to Organic Research > Position Changes. This is where you can browse through a timeline revealing how you've been gaining or losing keyword rankings across a period of time.
There, click on the particular dates that seem to have more going on and discover the specific keywords that were hit the most.
Editor's note: Asaad's suggestion to use SEMrush for this step is pretty on point, but our Positions Explorer here at Ahrefs offers a very similar set of features. To see your site's recent changes in organic keywords, just go to Positions Explorer, look up your site, and then go to Recent Changes (left sidebar).
I encourage you to check out both tools (SEMrush and Positions Explorer) and then just use the one that fits you better.
Final piece of advice. If the rankings are stable then you might want to check the overall search volume of your most important keywords. You may be dealing with a temporary glitch on the market itself. For example, if any new competing solutions or products have been released lately then their rising popularity may be the cause of your keywords dropping a bit.
4. Panguin Tool, for checking if you were hit by an algorithm update
This tool does a great job at showing any correlations between your search engine traffic and the known Google algorithm updates. All you have to do is go to the tool, log in with your GA account, and select any of your Analytics profiles.
This will present you with an overlay view of your organic traffic vs. Google algorithm updates. The great part here is that this works on any screen within GA, so you can look into the overall numbers, individual pieces of content, devices, and so on.
If you can see any correlations, look into a given Google update more closely and figure out what the problem is and what you can do to solve it.
We actually published some articles that can help you with that part:
- How to Identify and Recover from a Google Panda Penalty
- Google Panda 4.0: Initial Trends and Thoughts
- 7 Tips to Improving Your Local Presence Post Penguin in 2015
- How to Recover from a Google Penguin Penalty Using Ahrefs
5. Ahrefs, for discovering lost links
For this part of the job, we're going to use the Site Explorer.
While viewing your site's numbers, right in the center of the screen, you can see a nice chart presenting your New & Lost Backlinks:
If anything looks odd, you can go to Links - New / Lost (left menu, under Inbound Links) to do further investigation.
Once there, select the troublesome week, and look into the links you lost during that time:
At this moment, rather than just focusing on the gains and losses one by one, go a step further and aggregate the data into a pie chart. This will show you what areas of your site lost the most links.
Next, see if the data correlates to what you've gathered so far - mainly your GA data.
As we all know, links are not created equally, and sometimes losing one good .edu link can cause a significant drop, while losing 100 comment links goes unnoticed (or might even give you a boost).
Some possible reasons why you might lose a link:
- Your partnership with the site has ended and you've forgotten about it.
- The links you've earned got pushed down due to new content being created on the site.
- The link was manually removed by the editor.
- Technical difficulties.
6. SimilarWeb, for monitoring your competition
Things going slow for you isn't always something you can blame on algorithms, your links vanishing overnight, or the market in general. Sometimes, your competition is simply moving faster than you.
With SimilarWeb, you can check if that's indeed the case. Go to the main tool and look up your own site vs. your top 3-5 competitors. You can adjust the date range to only display the period that you've identified earlier through GA as the moment when you started losing your traffic.
Again, see if there's a correlation between you losing visibility and your competitors gaining some.
(Hint. At this point, you can also come back to Ahrefs and try identifying the links that your competitors gained while you were losing yours.)
7. Wayback Machine, for looking into your competitors' content
If your competitors indeed experienced a gain, apart from checking their backlinks (can be done with Ahrefs Site Explorer), you should also look into the content they either published recently or the past content that they fixed/improved.
Going back to SEMrush, you can use the tool to identify your competitors' top content in relation to the keywords they're ranking for. Once you have that, first check the content manually, just to see what it looks like and assess the value it brings to the visitor. Next, go to the Wayback Machine and see what this content used to look like prior to it gaining more SEO visibility.
Oftentimes, you will find that your competitors work really hard on improving past content. Having this sort of knowledge is a great starting point to figuring out what you can do to improve your own content game.
8. Performing quick technical checks
Okay, I know this final entry does stick out a bit since it's not a tool, but it's still a very important part of the process. After all, you don't want your efforts to all go to waste simply because you've overlooked a very basic issue with your site.
So, see into these and make sure there's nothing fishy going on with your site from a technical point of view:
- Check your robots.txt file and make sure you're not blocking any URLs or bots by accident.
- Check your .htaccess file for similar issues.
- Go back to GSC and see if the number of 404 pages on your site hasn't increased. Also, check if there isn't any drop in the number of your indexed pages overall.
- If you're using hard redirects, make sure they're code 301 - seen as permanent by the search engines.
- Check your canonical URLs. Are they pointing to the right place?
- Make sure you don't have competing pages trying to rank for the same keyword. Rather join those pages into one, stronger page.
Any of the above problems can occur for a number of reasons. For instance, maybe you installed some new plugins, or a new theme, or tried a new CDN service, or, even worse, your site got hacked. This is why doing a technical check is always time well spent.
That pretty much sums up the list of tools and the whole guide on what can be done if you experience any sort of SEO visibility/ranking problems. At this point you should know:
- what the areas of your site are that got affected the most,
- what the likely reasons are,
- whether or not your competitors played a role,
- how to fix the situation.
So what's your take here? Did we miss any important steps?