{"id":162553,"date":"2023-07-09T21:11:55","date_gmt":"2023-07-10T02:11:55","guid":{"rendered":"https:\/\/ahrefs.com\/blog\/?p=162553"},"modified":"2024-03-18T07:26:20","modified_gmt":"2024-03-18T12:26:20","slug":"blocked-robots-test","status":"publish","type":"post","link":"https:\/\/ahrefs.com\/blog\/blocked-robots-test\/","title":{"rendered":"The Story of Blocking 2 High-Ranking Pages With Robots.txt"},"content":{"rendered":"\n
I blocked two of our ranking pages using robots.txt. We lost a position here or there and all of the featured snippets for the pages. I expected a lot more impact, but the world didn\u2019t end.<\/div>\n\n\n\n
Warning<\/div>
\n\n\n\n

I don\u2019t recommend doing this, and it\u2019s entirely possible that your results may be different from ours.<\/p>\n\n\n\n<\/div><\/div>\n\n\n\n

I was trying to see the impact on rankings and traffic that the removal of content would have. My theory was that if we blocked the pages from being crawled, Google would have to rely on the link signals alone to rank the content.<\/p>\n\n\n\n

However, I don\u2019t think what I saw was actually the impact of removing the content. Maybe it is, but I can\u2019t say that with 100% certainty, as the impact feels too small. I\u2019ll be running another test to confirm this. My new plan is to delete the content from the page and see what happens.<\/p>\n\n\n\n

My working theory is that Google may still be using the content it used to see on the page to rank it. Google Search Advocate John Mueller<\/a> has confirmed this behavior in the past.<\/p>\n\n\n\n

\"John<\/figure>\n\n\n\n

So far, the test has been running for nearly five months. At this point, it doesn\u2019t seem like Google will stop ranking the page. I suspect, after a while, it will likely stop trusting that the content that was on the page is still there, but I haven\u2019t seen evidence of that happening.<\/p>\n\n\n\n

Keep reading to see the test setup and impact. The main takeaway is that accidentally blocking pages (that Google already ranks) from being crawled using robots.txt probably isn\u2019t going to have much impact on your rankings, and they will likely still show in the search results.<\/p>\n\n\n\n

Contents<\/div><\/span><\/span><\/span><\/a><\/div>\n\n\n\n