{"id":31336,"date":"2019-11-05T12:56:24","date_gmt":"2019-11-05T20:56:24","guid":{"rendered":"https:\/\/ahrefs.com\/blog\/?p=31336"},"modified":"2024-07-30T12:19:56","modified_gmt":"2024-07-30T17:19:56","slug":"duplicate-content","status":"publish","type":"post","link":"https:\/\/ahrefs.com\/blog\/duplicate-content\/","title":{"rendered":"Duplicate Content: Why It Happens and How to Fix&nbsp;It"},"content":{"rendered":"<div class=\"intro-txt\">Looking to learn what duplicate content is, and how it might be hurting your&nbsp;SEO?&nbsp;<\/div>\n<p>Duplicate content is a source of constant anxiety for many site owners.<\/p>\n<p>Read almost anything about it, and you\u2019ll come away believing that your site is a ticking time bomb of duplicate content issues. A Google penalty is merely days&nbsp;away.<\/p>\n<p>Thankfully, this isn\u2019t true\u2014but duplicate content can still cause SEO issues. And with <a href=\"https:\/\/www.youtube.com\/watch?v=mQZY7EmjbMA&amp;feature=youtu.be&amp;t=19\" target=\"_blank\" rel=\"noopener noreferrer\">25-30%<\/a>&nbsp;of the web being duplicate content, it\u2019s useful to know how to avoid and fix such issues.<\/p>\n<p>In this guide, you\u2019ll learn:<\/p>\n<ul>\n<li><a href=\"#what-is-duplicate-content\">What duplicate content is<\/a>;<\/li>\n<li><a href=\"#why-is-duplicate-content-bad-for-seo\">Why duplicate content is bad for SEO<\/a>;<\/li>\n<li><a href=\"#does-google-have-a-duplicate-content-penalty\">Whether Google has a duplicate content penalty<\/a>;<\/li>\n<li><a href=\"#duplicate-content-causes\">Common causes of duplicate content<\/a>;<\/li>\n<li><a href=\"#how-to-check-for-duplicate-content\">How to check for (and fix) duplicate content<\/a><\/li>\n<\/ul>\n<div class=\"hub-link\"><img decoding=\"async\" alt=\"Beginner's guide to technical SEO\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/svg\/4.svg\"><div class=\"hl-title\">New to technical SEO? Check out&nbsp;our<\/div><div class=\"hl-content\"><a href=\"https:\/\/ahrefs.com\/blog\/technical-seo\/\" target=\"_blank\">Beginner\u2019s guide to technical SEO<\/a><\/div><\/div>\n<h2 id=\"what-is-duplicate-content\">What is duplicate content?<\/h2>\n<p>Duplicate content is the same or similar content that appears on the web in more than one place. It can exist on one website or across multiple websites.<\/p>\n<p>For example, let\u2019s play a game of spot the difference\u2026<\/p>\n<p>Here\u2019s the page located at caltonnutrition.com\/tag\/protein-powder\/\u2026<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1.png\"><\/p>\n<p>\u2026 and here\u2019s the page located at caltonnutrition.com\/tag\/whey\/:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1.png\"><\/p>\n<p>Notice any difference? Me neither. The content at both URLs is identical. AKA <em>duplicate content<\/em>.<\/p>\n<h2 id=\"why-is-duplicate-content-bad-for-seo\">Why is duplicate content bad for&nbsp;SEO?<\/h2>\n<p>Google <a href=\"https:\/\/developers.google.com\/search\/blog\/2008\/09\/demystifying-duplicate-content-penalty\">says<\/a> there\u2019s no such thing as a duplicate content penalty. But duplicate content <em>can<\/em> hurt your SEO performance for a few reasons.<\/p>\n<ol>\n<li>Undesirable or unfriendly URLs in search results;<\/li>\n<li>Backlink dilution;<\/li>\n<li>Burns crawl budget;<\/li>\n<li>Scraped or syndicated content outranking you.<\/li>\n<\/ol>\n<h3>1. Undesirable or unfriendly URLs in search results<\/h3>\n<p>Imagine that the same page is available at three different URLs:<\/p>\n<ol>\n<li>domain.com\/page\/<\/li>\n<li>domain.com\/page\/?utm_content=buffer&amp;utm_medium=social<\/li>\n<li>domain.com\/category\/page\/<\/li>\n<\/ol>\n<p>The first should show up in search results, but Google can get this wrong. If that happens, an undesirable URL may take its&nbsp;place.<\/p>\n<p>Because people may be less inclined to click on an unfriendly URL, you may get less organic traffic.<\/p>\n<h3>2. Backlink dilution<\/h3>\n<p>If the same content is available at many URLs, then each of those URLs may attract backlinks. That results in the splitting of \u201clink equity\u201d between URLs.<\/p>\n<p>To show an example of this in the wild, take a look at these two pages on <em>buffer.com<\/em>:<\/p>\n<blockquote><p>https:\/\/buffer.com\/<strong>library<\/strong>\/social-media-manager-checklist<br>\nhttps:\/\/buffer.com\/<strong>resources<\/strong>\/social-media-manager-checklist<\/p><\/blockquote>\n<p>These pages are almost exact duplicates. And they have 106 and 144 referring domains (links from unique websites), respectively.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"900\" height=\"211\" class=\"wp-image-31335\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/buffer-url-1-referring-domains.png\" alt=\"buffer url 1 referring domains\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/buffer-url-1-referring-domains.png 900w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/buffer-url-1-referring-domains-768x180.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/buffer-url-1-referring-domains-680x159.png 680w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\"><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"900\" height=\"214\" class=\"wp-image-31327\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/buffer-url-2-referring-domains.png\" alt=\"buffer url 2 referring domains\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/buffer-url-2-referring-domains.png 900w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/buffer-url-2-referring-domains-768x183.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/buffer-url-2-referring-domains-680x162.png 680w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\"><\/p>\n<p>Before you panic, know that this isn\u2019t always a problem because of how Google handles duplicate content.<\/p>\n<p>In simple terms, when they detect duplicate content, <a href=\"https:\/\/webmasters.googleblog.com\/2007\/09\/google-duplicate-content-caused-by-url.html\" target=\"_blank\" rel=\"noopener noreferrer\">they group the URLs into one cluster<\/a>. They then \u201cselect what [they] think is the \u2018best\u2019 URL to represent the cluster in search results\u201d and \u201cconsolidate properties of the URLs in the cluster, <strong>such as link popularity<\/strong>, to the representative URL.\u201d This process is known as <a href=\"https:\/\/ahrefs.com\/blog\/canonicalization\/\">canonicalization<\/a>.<\/p>\n<p>So, in the case above, Google <em>should<\/em>&nbsp;show only one of the URLs in organic search and attribute all referring domains in the cluster (106+144) to that&nbsp;URL.<\/p>\n<p>But that\u2019s not what happens, as we see both URLs ranking in Google for similar keywords.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"900\" height=\"209\" class=\"wp-image-31332\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/google-result-buffer-1.png\" alt=\"google result buffer 1\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/google-result-buffer-1.png 900w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/google-result-buffer-1-768x178.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/google-result-buffer-1-680x158.png 680w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\"><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"900\" height=\"223\" class=\"wp-image-31323\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/google-result-buffer-2.png\" alt=\"google result buffer 2\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/google-result-buffer-2.png 900w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/google-result-buffer-2-768x190.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/google-result-buffer-2-680x168.png 680w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\"><\/p>\n<p>In this instance, Google likely isn\u2019t consolidating \u201clink equity\u201d at one&nbsp;URL.<\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">DISCLAIMER<\/div><div class=\"recommendation-content\">\n<p>We can\u2019t be sure how Google sees these two URLs, as we don\u2019t have access to Buffer\u2019s Google Search Console account. It may be that they see both of these URLs as duplicates, and one of them will disappear from organic search soon.<\/p>\n<\/div><\/div>\n<h3>3. Burns crawl budget<\/h3>\n<p>Google finds new content on your website via crawling, which means they follow links from existing pages to new pages. They also recrawl pages they know about from time to time to see if anything has changed.<\/p>\n<p>Having duplicate content serves only to create more work for them. That can affect the speed and frequency at which they crawl your new or updated pages.<\/p>\n<p>That\u2019s bad because it may lead to delays in indexing new pages and reindexing updated pages.<\/p>\n<div class=\"sidenote\"><div class=\"sidenote-title\">Sidenote.<\/div>&nbsp;Because <a href=\"https:\/\/ahrefs.com\/blog\/crawl-budget\/\" target=\"_blank\" rel=\"noopener noreferrer\">Google\u2019s \u201ccrawl rate limit<\/a>\u201d is higher for more responsive websites, this is more of an issue for slow websites with smaller bandwidth allowances. Their systems will also <a href=\"https:\/\/support.google.com\/webmasters\/answer\/139066?hl=en\" target=\"_blank\" rel=\"noopener noreferrer\">crawl the duplicate URLs less often<\/a>.<\/div>\n<h3>4. Scraped content outranking you<\/h3>\n<p>Occasionally, you may permit another website to republish your content. That\u2019s known as syndication. Other times, sites may scrape your content and republish it without permission.<\/p>\n<p>Both of these scenarios lead to duplicate content across multiple domains, but they usually don\u2019t cause problems. It\u2019s only when the scraped or republished content starts outranking the original on your site that issues arise.<\/p>\n<p>The good news is this is a rare occurrence, but it can happen.<\/p>\n<h2 id=\"does-google-have-a-duplicate-content-penalty\">Does Google have a duplicate content penalty?<\/h2>\n<p>Google has stated on multiple occasions that they don\u2019t have a duplicate content penalty.<\/p>\n<blockquote class=\"small\"><div class=\"quote-content\"> We don\u2019t have a duplicate content penalty. It\u2019s not that we would demote a site for having a lot of duplicate content.&nbsp;<\/div><div class=\"quote-info clearfix\"><div class=\"quote-photo\"><img decoding=\"async\" alt=\"John Mueller\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2018\/01\/john-mueller.png\"><\/div><div class=\"extra-box\"><span class=\"quote-author\">John Mueller,<\/span> <span class=\"quote-author-job\">Webmaster Trends Analyst <a href=\"https:\/\/www.youtube.com\/watch?v=s7kP7LWM87Y&amp;feature=youtu.be&amp;t=6m33s\" target=\"_blank\">Google<\/a><\/span><\/div><\/div><\/blockquote>\n<blockquote class=\"small\"><div class=\"quote-content\"> Let\u2019s put this to bed once and for all, folks: There\u2019s no such thing as a duplicate content penalty.&nbsp;<\/div><div class=\"quote-info clearfix\"><div class=\"quote-photo\"><img decoding=\"async\" alt=\"Susan Moskwa\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/susan-moskwa.png\"><\/div><div class=\"extra-box\"><span class=\"quote-author\">Susan Moskwa,<\/span> <span class=\"quote-author-job\">Former Webmaster Trends Analyst <a href=\"https:\/\/webmasters.googleblog.com\/2008\/09\/demystifying-duplicate-content-penalty.html\" target=\"_blank\">Google<\/a><\/span><\/div><\/div><\/blockquote>\n<blockquote class=\"small\"><div class=\"quote-content\"> DYK Google doesn\u2019t have a duplicate content penalty.&nbsp;<\/div><div class=\"quote-info clearfix\"><div class=\"quote-photo\"><img decoding=\"async\" alt=\"Gary Illyes\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2018\/02\/google-gary-illyes.jpg\"><\/div><div class=\"extra-box\"><span class=\"quote-author\">Gary Illyes,<\/span> <span class=\"quote-author-job\">Webmaster Trends Analyst <a href=\"https:\/\/twitter.com\/methode\/status\/831047566284431360\" target=\"_blank\">Google<\/a><\/span><\/div><\/div><\/blockquote>\n<p>But, this isn\u2019t entirely true. If your duplicate content is accidental and not the result of intentional manipulation of search results or spammy practices, then you won\u2019t get penalized. If it is, then you&nbsp;might.<\/p>\n<p>Google confirms that <a href=\"https:\/\/support.google.com\/webmasters\/answer\/66359?hl=en\" target=\"_blank\" rel=\"noopener noreferrer\">here<\/a>:<\/p>\n<blockquote><p>In the rare cases in which Google perceives that duplicate content may be shown with intent to manipulate our rankings and deceive our users, we\u2019ll also make appropriate adjustments in the indexing and ranking of the sites involved. As a result, the ranking of the site may suffer, or the site might be removed entirely from the Google index, in which case it will no longer appear in search results.<\/p><\/blockquote>\n<p>The question is, what counts as \u201cintent to manipulate our rankings and deceive our&nbsp;users\u201d?<\/p>\n<p>Google has a lot of information on that <a href=\"https:\/\/support.google.com\/webmasters\/answer\/9044175?visit_id=636973311554955646-4156206938&amp;rd=2#thin-content\" target=\"_blank\" rel=\"noopener noreferrer\">here<\/a>. But basically, it\u2019s things like:<\/p>\n<ul>\n<li>Intentionally creating multiple pages, subdomains, or domains with lots of duplicate content.<\/li>\n<li>Publishing lots of scraped content<\/li>\n<li>Publishing affiliate content scraped from Amazon or other sites (and adding no additional value)<\/li>\n<\/ul>\n<p>However, as discussed above, duplicate content can still hurt SEO\u2014even without a penalty.<\/p>\n<h2 id=\"duplicate-content-causes\">Common causes of duplicate content<\/h2>\n<p>There\u2019s no single cause of duplicate content. There are&nbsp;many.<\/p>\n<h3>Faceted\/filtered navigation<\/h3>\n<p>Faceted navigation is where users can filter and sort items on the page. Ecommerce websites use it a&nbsp;lot.<\/p>\n<p>This kind of navigation appends <a href=\"https:\/\/ahrefs.com\/blog\/url-parameters\/\">parameters<\/a> to the end of the&nbsp;URL.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"449\" height=\"36\" class=\"wp-image-31330\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/faceted-navigation-url.png\" alt=\"faceted navigation url\"><\/p>\n<p>Because there are usually many combinations of these filters, faceted navigation often results in lots of duplicate-or-near-duplicate content.<\/p>\n<p>Take a look at these two pages, for example:<\/p>\n<blockquote><p>bbclothing.co.uk\/en-gb\/clothing\/shirts.html?new_style=Checked<br>\nbbclothing.co.uk\/en-gb\/clothing\/shirts.html?Size=S&amp;new_style=Checked<\/p><\/blockquote>\n<p>The URLs are unique, but the content is almost identical.<\/p>\n<p>Plus, the order of the parameters often doesn\u2019t matter. For example, the same page is accessible at both of these&nbsp;URLs:<\/p>\n<blockquote><p>bbclothing.co.uk\/en-gb\/clothing\/shirts.html?new_style=Checked&amp;Size=XL<br>\nbbclothing.co.uk\/en-gb\/clothing\/shirts.html?Size=XL&amp;new_style=Checked<\/p><\/blockquote>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Faceted navigation is a complex beast. If you suspect this as the cause of your duplicate content issues, just <a href=\"https:\/\/builtvisible.com\/faceted-navigation-seo-best-practices\/\" target=\"_blank\" rel=\"noopener noreferrer\">read this<\/a>.<\/p>\n<\/div><\/div>\n<h3>Tracking parameters<\/h3>\n<p>Parameterized URLs are also used for tracking purposes. For example, you may use UTM parameters to track visits from a newsletter campaign in Google Analytics:<\/p>\n<p><strong>Example: <\/strong>example.com\/page?utm_source=newsletter<\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p><a href=\"https:\/\/ahrefs.com\/blog\/canonical-tags\/\">Canonicalize<\/a>&nbsp;your parameterized URLs to SEO-friendly versions without tracking parameters.<\/p>\n<\/div><\/div>\n<h3>Session IDs<\/h3>\n<p>Session IDs store information about your visitors. They usually append a long string to the URL like&nbsp;so:<\/p>\n<p><strong>Example: <\/strong>example.com?sessionId=jow8082345hnfn9234<\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p><a href=\"https:\/\/ahrefs.com\/blog\/canonical-tags\/\">Canonicalize<\/a>&nbsp;the URLs to SEO-friendly versions.<\/p>\n<\/div><\/div>\n<h3>HTTPS vs. HTTP, and non-www vs.&nbsp;www<\/h3>\n<p>Most websites are accessible at one of these four variations:<\/p>\n<ul>\n<li><strong>https:\/\/www.<\/strong>example.com (HTTPS, www)<\/li>\n<li><strong>https:\/\/<\/strong>example.com (HTTPS, non-www)<\/li>\n<li><strong>http:\/\/www.<\/strong>example.com (HTTP, www)<\/li>\n<li><strong>http:\/\/<\/strong>example.com (HTTP, non-www)<\/li>\n<\/ul>\n<p>If you\u2019re using HTTPS, it\u2019ll be one of the first two. Whether it\u2019s the www or non-www version is your choice.<\/p>\n<p>However, if you don\u2019t correctly configure your server, your site will be accessible at two or more of these variations. That isn\u2019t good and can lead to duplicate content issues.<\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Use <a href=\"https:\/\/ahrefs.com\/blog\/301-redirects\/\">redirects<\/a>&nbsp;to ensure that your website is only accessible at one location.<\/p>\n<\/div><\/div>\n<h3>Case-sensitive URLs<\/h3>\n<p>Google sees URLs as case-sensitive.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-172962\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/Screenshot-2024-03-18-at-12.33.19.png\" alt=\"John Mueller confirms that Google sees URLs as case sensitive\" width=\"1160\" height=\"406\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/Screenshot-2024-03-18-at-12.33.19.png 1160w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/Screenshot-2024-03-18-at-12.33.19-680x238.png 680w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/Screenshot-2024-03-18-at-12.33.19-768x269.png 768w\" sizes=\"auto, (max-width: 1160px) 100vw, 1160px\"><\/p>\n<div class=\"sidenote\"><div class=\"sidenote-title\">Sidenote.<\/div>&nbsp;This <a href=\"https:\/\/www.propellernet.co.uk\/search-engines-treat-trailing-slashes-capital-letters-urls\/\" target=\"_blank\" rel=\"noopener noreferrer\">doesn\u2019t seem<\/a>&nbsp;to be the case for Bing, which treats all URLs as lowercase.&nbsp;<\/div>\n<p>That means these three URLs are all different:<\/p>\n<ul>\n<li>example.com\/page<\/li>\n<li>example.com\/PAGE<\/li>\n<li>example.com\/pAgE<\/li>\n<\/ul>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Be consistent with internal links (i.e., don\u2019t internally link to multiple versions of URLs). If that doesn\u2019t solve things, you can always canonicalize or redirect.<\/p>\n<\/div><\/div>\n<h3>Trailing slashes vs. non-trailing-slashes<\/h3>\n<p>Google treats <a href=\"https:\/\/ahrefs.com\/blog\/trailing-slash\/\">URLs with and without trailing slashes<\/a> as unique. That means these two URLs are unique in Google\u2019s eyes:<\/p>\n<ul>\n<li>example.com\/page\/<\/li>\n<li>example.com\/page<\/li>\n<\/ul>\n<p>If your content is accessible at both URLs, then that can lead to duplicate content issues.<\/p>\n<p>To check if this is an issue, try to load a page with and without the trailing slash. Ideally, only one version will load. The other will redirect.<\/p>\n<p>For example, if you try to load this post without the trailing slash, it will redirect to the URL with the trailing slash.<\/p>\n<p>Google states that this behavior is <a href=\"https:\/\/webmasters.googleblog.com\/2010\/04\/to-slash-or-not-to-slash.html\" target=\"_blank\" rel=\"noopener noreferrer\">ideal<\/a>.<\/p>\n<blockquote><p>If only one version can be returned (i.e., the other redirects to it), that\u2019s great! This behavior is beneficial because it reduces duplicate content.<\/p><\/blockquote>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Redirect the undesirable version (e.g., without trailing slash) to the desired version (e.g., with trailing slash). You should also make sure to stay consistent with <a href=\"https:\/\/ahrefs.com\/blog\/prioritize-internal-linking\/\">internal linking.<\/a> Don\u2019t link to versions with trailing slashes sometimes, and without other times. Choose one and stick with&nbsp;it.<\/p>\n<\/div><\/div>\n<h3>Print-friendly URLs<\/h3>\n<p>Print-friendly versions have the same content as the original. It\u2019s only the URL that differs.<\/p>\n<ul>\n<li>example.com\/page<\/li>\n<li>example.com\/<strong>print<\/strong>\/page<\/li>\n<\/ul>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Canonicalize the print-friendly version to the original.<\/p>\n<\/div><\/div>\n<h3>Mobile-friendly URLs<\/h3>\n<p>Mobile-friendly URLs, like print-friendly URLs, are duplicates.<\/p>\n<ul>\n<li>example.com\/page<\/li>\n<li><strong>m.<\/strong>example.com\/page<\/li>\n<\/ul>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Canonicalize the mobile-friendly version to the original. Use rel=\u201calternate\u201d to tell Google that the mobile-friendly URL is an alternate version of the desktop content.<\/p>\n<p><strong>Recommended reading:<\/strong>&nbsp;<em><a href=\"https:\/\/developers.google.com\/search\/mobile-sites\/mobile-seo\/separate-urls#annotations-for-desktop-and-mobile-urls\" target=\"_blank\" rel=\"noopener noreferrer\">Annotations for desktop and mobile URLs<\/a><\/em><\/p>\n<\/div><\/div>\n<h3>AMP URLs<\/h3>\n<p>Accelerated Mobile Pages (AMP) are duplicates.<\/p>\n<ul>\n<li>example.com\/page<\/li>\n<li>example.com\/<strong>amp<\/strong>\/page<\/li>\n<\/ul>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Canonicalize the AMP version to the non-AMP version. Use <code>rel=\"amphtml\"<\/code> to tell Google that the AMP URL is an alternate version of the non-AMP content.<\/p>\n<p>If you only have AMP content, use a self-referencing canonical tag.<\/p>\n<p><strong>Recommended reading:<\/strong>&nbsp;<em><a href=\"https:\/\/amp.dev\/documentation\/guides-and-tutorials\/optimize-and-measure\/discovery\/\" target=\"_blank\" rel=\"noopener noreferrer\">Make your pages discoverable - amp.dev<\/a><\/em><\/p>\n<\/div><\/div>\n<h3>Tag and category pages<\/h3>\n<p>Most CMS\u2019 create dedicated tags pages when you use&nbsp;tags.<\/p>\n<p>For example, if you have an article about organic whey protein, and you use both \u201cprotein powder\u201d and \u201cwhey\u201d as tags, then you\u2019ll end up with two tag pages like&nbsp;these:<\/p>\n<blockquote><p>https:\/\/www.caltonnutrition.com\/tag\/whey\/<br>\nhttps:\/\/www.caltonnutrition.com\/tag\/protein-powder\/<\/p><\/blockquote>\n<p>That doesn\u2019t always cause duplicate content in itself, but it&nbsp;can.<\/p>\n<p>That\u2019s the case here because there\u2019s only one page on the site with those two tags\u2014so each tag page is identical.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"900\" height=\"541\" class=\"wp-image-31333\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1.png\" alt=\"duplicate tag 1\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1.png 900w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1-768x462.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1-680x409.png 680w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\"><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"900\" height=\"541\" class=\"wp-image-31333\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1.png\" alt=\"duplicate tag 1\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1.png 900w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1-768x462.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-tag-1-680x409.png 680w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\"><\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Two options:<\/p>\n<ol>\n<li><strong>Don\u2019t use tags.<\/strong>&nbsp;Most of the time, they have little to no value anyway.<\/li>\n<li><strong>Noindex your tags pages. <\/strong>This doesn\u2019t solve the issue of crawl budget, as Google will still waste time crawling these&nbsp;pages.<\/li>\n<\/ol>\n<p>Note that category pages can cause similar issues to tags pages. Case in&nbsp;point:<\/p>\n<blockquote><p>https:\/\/www.xs-stock.co.uk\/adidas\/<br>\nhttps:\/\/www.xs-stock.co.uk\/brands\/Chelsea-FC.html<\/p><\/blockquote>\n<p>Both of these pages are almost identical because there are no products listed under either category. So all we\u2019re left with is the boilerplate template copy.<\/p>\n<p>Solve this by using a reasonable number of categories on your site, or even <a href=\"https:\/\/ahrefs.com\/blog\/meta-robots\/\">noindexing<\/a>&nbsp;your category pages.<\/p>\n<\/div><\/div>\n<h3>Attachment image&nbsp;URLs<\/h3>\n<p>Many CMS\u2019 create dedicated pages for image attachments. These pages usually show nothing but the image and some boilerplate copy.<\/p>\n<p>Because this copy is the same across all auto-generated pages, it leads to duplicate content.<\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Disable dedicated pages for images in your CMS. In WordPress, you can do this using <a href=\"https:\/\/ahrefs.com\/blog\/best-seo-plugins-for-wordpress\/\">a plugin<\/a>&nbsp;like Yoast.<\/p>\n<\/div><\/div>\n<h3>Paginated comments<\/h3>\n<p>WordPress and other CMS\u2019 allow for paginated comments. This causes duplicate content as it effectively creates multiple versions of the same&nbsp;URLs.<\/p>\n<ul>\n<li>example.com\/post\/<\/li>\n<li>example.com\/post\/comment-page-2<\/li>\n<li>example.com\/post\/comment-page-3<\/li>\n<\/ul>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Turn off comment pagination or noindex your <a href=\"https:\/\/ahrefs.com\/blog\/rel-prev-next-pagination\/\">paginated pages<\/a> using a plugin like&nbsp;Yoast.<\/p>\n<\/div><\/div>\n<h3>Localization<\/h3>\n<p>If you\u2019re serving similar content to people in different locales who speak the same language, then that can cause duplicate content.<\/p>\n<p>For example, you might have different versions of your site for people in the US, UK, and Australia. Because there are likely only minor differences between the content served to each locale (e.g., prices in dollars versus pounds sterling), the versions will be near duplicates.<\/p>\n<div class=\"sidenote\"><div class=\"sidenote-title\">Sidenote.<\/div>&nbsp;According to John Mueller, <a href=\"https:\/\/youtu.be\/cxWo4ttPgAc?t=207\" target=\"_blank\" rel=\"noopener noreferrer\">translated content is not duplicate content<\/a>.<\/div>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Use <a href=\"https:\/\/ahrefs.com\/blog\/hreflang-tags\/\">hreflang tags<\/a>&nbsp;to tell search engines about the relationship between the variations.<\/p>\n<\/div><\/div>\n<h3>Search results pages<\/h3>\n<p>Lots of websites have search boxes. Using these typically takes you to a parameterized search URL.<\/p>\n<p><strong>Example: <\/strong>example.com?q=search-term<\/p>\n<p>Google\u2019s former Head of Webspam, Matt Cutts, <a href=\"https:\/\/www.mattcutts.com\/blog\/search-results-in-search-results\/\" target=\"_blank\" rel=\"noopener noreferrer\">stated<\/a>&nbsp;that:<\/p>\n<blockquote class=\"small\"><div class=\"quote-content\"> Typically, web search results don\u2019t add value to users, and since our core goal is to provide the best search results possible, we generally exclude search results from our web search index. (Not all URLs that contain things like \u201c\/results\u201d or \u201c\/search\u201d are search results, of course.)&nbsp;<\/div><div class=\"quote-info clearfix\"><div class=\"quote-photo\"><img decoding=\"async\" alt=\"Matt Cutts\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2015\/05\/Matt-Cutts.png\"><\/div><div class=\"extra-box\"><span class=\"quote-author\">Matt Cutts,<\/span> <span class=\"quote-author-job\">Former Head of Webspam <a href=\"https:\/\/www.mattcutts.com\/blog\/search-results-in-search-results\/\" target=\"_blank\">Google<\/a><\/span><\/div><\/div><\/blockquote>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Use a robots meta tag to <a href=\"https:\/\/ahrefs.com\/blog\/remove-urls-from-google\/\">remove search pages from Google\u2019s index<\/a> or&nbsp;block access to search results pages in <a href=\"https:\/\/ahrefs.com\/blog\/robots-txt\/\">robots.txt<\/a>. Refrain from internally linking to search results pages.<\/p>\n<\/div><\/div>\n<h3>Staging&nbsp;environment<\/h3>\n<p>A staging environment is a duplicate or near-duplicate version of your site used for testing purposes.<\/p>\n<p>For example, imagine that you want to install a new plugin or change some code on your website. You might not want to push that straight to a live site with hundreds of thousands of daily visitors. The risk of catastrophe is too high. The solution is to test the changes in a staging environment first.<\/p>\n<p>Staging environments become an SEO issue when Google indexes them because it results in duplicate content.<\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">How to solve this&nbsp;issue<\/div><div class=\"recommendation-content\">\n<p>Protect your staging environment using HTTP authentication, IP whitelisting, or VPN access. If it\u2019s already indexed, use a robots noindex directive to get it removed.<\/p>\n<\/div><\/div>\n<h2>How to check for duplicate content on your&nbsp;site<\/h2>\n<p>Head over to <a href=\"https:\/\/ahrefs.com\/site-audit\">Ahrefs\u2019 Site Audit<\/a>&nbsp;and start a&nbsp;crawl.<\/p>\n<div data-mode=\"normal\" data-oembed=\"1\" data-provider=\"youtube\" id=\"arve-youtube-ljinwqfgyve\" style=\"max-width:900px;\" class=\"arve\">\n<div class=\"arve-inner\">\n<div style=\"aspect-ratio:500\/281\" class=\"arve-embed arve-embed--has-aspect-ratio\">\n<div class=\"arve-ar\" style=\"padding-top:56.200000%\"><\/div>\n<p>\t\t\t<iframe allow=\"accelerometer 'none';autoplay 'none';bluetooth 'none';browsing-topics 'none';camera 'none';clipboard-read 'none';clipboard-write;display-capture 'none';encrypted-media 'none';gamepad 'none';geolocation 'none';gyroscope 'none';hid 'none';identity-credentials-get 'none';idle-detection 'none';keyboard-map 'none';local-fonts;magnetometer 'none';microphone 'none';midi 'none';otp-credentials 'none';payment 'none';picture-in-picture;publickey-credentials-create 'none';publickey-credentials-get 'none';screen-wake-lock 'none';serial 'none';summarizer 'none';sync-xhr;usb 'none';web-share;window-management 'none';xr-spatial-tracking 'none';\" allowfullscreen class=\"arve-iframe fitvidsignore\" credentialless data-arve=\"arve-youtube-ljinwqfgyve\" data-lenis-prevent data-src-no-ap=\"https:\/\/www.youtube-nocookie.com\/embed\/LjinWqfGyVE?feature=oembed&amp;iv_load_policy=3&amp;modestbranding=1&amp;rel=0&amp;autohide=1&amp;playsinline=0&amp;autoplay=0\" frameborder=\"0\" height=\"505.8\" loading=\"lazy\" name referrerpolicy=\"strict-origin-when-cross-origin\" sandbox=\"allow-scripts allow-same-origin allow-presentation allow-popups allow-popups-to-escape-sandbox\" scrolling=\"no\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/LjinWqfGyVE?feature=oembed&amp;iv_load_policy=3&amp;modestbranding=1&amp;rel=0&amp;autohide=1&amp;playsinline=0&amp;autoplay=0\" title width=\"900\"><\/iframe><\/p><\/div>\n<\/div>\n<\/div>\n<p>Once done, head to the <strong>Content quality<\/strong>&nbsp;report.<\/p>\n<p>Look for clusters of duplicates and near-duplicates without a canonical. These are highlighted in orange.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"900\" height=\"595\" class=\"wp-image-31331\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/ahrefs-site-audit-duplicates-without-canonical.png\" alt=\"ahrefs site audit duplicates without canonical\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/ahrefs-site-audit-duplicates-without-canonical.png 900w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/ahrefs-site-audit-duplicates-without-canonical-768x508.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/ahrefs-site-audit-duplicates-without-canonical-643x425.png 643w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\"><\/p>\n<p>Click any of these clusters to see the affected pages.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"900\" height=\"478\" class=\"wp-image-31326\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-pages-without-canonicals.png\" alt=\"duplicate pages without canonicals\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-pages-without-canonicals.png 900w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-pages-without-canonicals-768x408.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/duplicate-pages-without-canonicals-680x361.png 680w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\"><\/p>\n<p>Investigate the reason for the duplicate content, then take the appropriate action.<\/p>\n<p>Note that these won\u2019t always be issues that need rectifying, especially in the case of near duplicates.<\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">Not an Ahrefs user?<\/div><div class=\"recommendation-content\">\n<p>Look for these duplicate-content-related warnings in Google Search Console:<\/p>\n<ul>\n<li>Duplicate without user-selected canonical<\/li>\n<li>Duplicate, Google chose different canonical than&nbsp;user<\/li>\n<li>Duplicate, submitted URL not selected as canonical<\/li>\n<\/ul>\n<p>Learn more about how to deal with these warnings <a href=\"https:\/\/support.google.com\/webmasters\/answer\/7440203?hl=en\" target=\"_blank\" rel=\"noopener noreferrer\">here<\/a>.<\/p>\n<p>To see how Google treats a specific URL, use the <a href=\"https:\/\/support.google.com\/webmasters\/answer\/9012289?hl=en\" target=\"_blank\" rel=\"noopener noreferrer\">URL Inspection tool<\/a>.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"879\" height=\"499\" class=\"wp-image-31324\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/url-inspection.png\" alt=\"url inspection\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/url-inspection.png 879w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/url-inspection-768x436.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/url-inspection-680x386.png 680w\" sizes=\"auto, (max-width: 879px) 100vw, 879px\"><\/p>\n<\/div><\/div>\n<p>You can also check for duplicate title tags, <a href=\"https:\/\/ahrefs.com\/blog\/meta-description\/\">meta descriptions<\/a>, and H1s in the <strong>HTML tags <\/strong>report.<\/p>\n<p>Bad duplicates are what you\u2019re looking for. These are pages with duplicate meta tags but different canonicals.<\/p>\n<p>Select these by clicking the \u201cBad duplicates\u201d toggle under <strong>HTML tags &amp; content.<\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"845\" height=\"335\" class=\"wp-image-31334\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/bad-duplicates-html-tags-site-audit.png\" alt=\"bad duplicates html tags site audit\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/bad-duplicates-html-tags-site-audit.png 845w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/bad-duplicates-html-tags-site-audit-768x304.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/bad-duplicates-html-tags-site-audit-680x270.png 680w\" sizes=\"auto, (max-width: 845px) 100vw, 845px\"><\/p>\n<p>Click on any of the yellow bars to see the affected pages.<\/p>\n<p>Pages with duplicate titles, meta descriptions, or H1\u2019s are often very similar.<\/p>\n<p>For example, these two have the same title tag, and the content is almost identical because the product is the same. The only difference is that one of the pages is for a 3-pack of instant lighting firelogs, whereas the other is for just&nbsp;one.<\/p>\n<blockquote><p>https:\/\/www.xs-stock.co.uk\/big-k-instant-light-the-wrapper-firelog-3-pack-camp-fire-fuel\/<br>\nhttps:\/\/www.xs-stock.co.uk\/big-k-instant-light-the-wrapper-firelog-camp-fire-chiminea\/<\/p><\/blockquote>\n<p>Google <a href=\"https:\/\/support.google.com\/webmasters\/answer\/66359?hl=en\" target=\"_blank\" rel=\"noopener noreferrer\">states<\/a>&nbsp;that you should minimise similar content like&nbsp;this:<\/p>\n<blockquote class=\"small\"><div class=\"quote-content\"> If you have many pages that are similar, consider expanding each page or consolidating the pages into&nbsp;one.&nbsp;<\/div><\/blockquote>\n<p>However, a small number of similar pages is unlikely to be much of an&nbsp;issue.<\/p>\n<h2 id=\"how-to-check-for-duplicate-content\">How to check for duplicate content issues across the&nbsp;web<\/h2>\n<p>Content scraping and syndication can also lead to duplicate content issues. But it\u2019s only usually an issue if you see scraped versions of your content outranking you.<\/p>\n<p>Does that happen? Yes, but it\u2019s often more of an issue for new or weak websites. Why? Because the sites scraping your content are often more authoritative. That sometimes \u201ctricks\u201d Google into thinking that theirs is the original.<\/p>\n<p>If you have a small website, then you can often find scraped content by searching Google for a snippet of text from your page in quotes.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"788\" height=\"384\" class=\"wp-image-31329\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/scraped-content-google.png\" alt=\"scraped content google\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/scraped-content-google.png 788w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/scraped-content-google-768x374.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/scraped-content-google-680x331.png 680w\" sizes=\"auto, (max-width: 788px) 100vw, 788px\"><\/p>\n<p>For larger sites, you\u2019ll need to use an automated tool like <a href=\"https:\/\/www.copyscape.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Copyscape<\/a>. This searches the web for other occurrences of the content on your page(s).<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"714\" height=\"601\" class=\"wp-image-31325\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/copyscape.png\" alt=\"copyscape\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/copyscape.png 714w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/copyscape-505x425.png 505w\" sizes=\"auto, (max-width: 714px) 100vw, 714px\"><\/p>\n<p>Whichever method you use, most results will be from spammy and low-quality sites.<\/p>\n<p>Generally speaking, these are nothing to worry about. However, if you see that a legitimate website scraped your content, and are concerned that it may be stealing your traffic, throw the URL into <a href=\"https:\/\/ahrefs.com\/site-explorer\">Ahrefs\u2019 Site Explorer<\/a>&nbsp;to see an organic traffic estimate.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"938\" height=\"191\" class=\"wp-image-31328\" src=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/ahrefs-traffic-estimate.png\" alt=\"ahrefs traffic estimate\" srcset=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/ahrefs-traffic-estimate.png 938w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/ahrefs-traffic-estimate-768x156.png 768w, https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/ahrefs-traffic-estimate-680x138.png 680w\" sizes=\"auto, (max-width: 938px) 100vw, 938px\"><\/p>\n<p>If it\u2019s getting more traffic than your page, then there may be an&nbsp;issue.<\/p>\n<p>In this case, you have three options:<\/p>\n<ol>\n<li>Reach out and request that they remove the content.<\/li>\n<li>Reach out and request they add a canonical link to the original on your&nbsp;site.<\/li>\n<li><a href=\"https:\/\/support.google.com\/legal\/troubleshooter\/1114905?hl=en\" target=\"_blank\" rel=\"noopener noreferrer\">Submit a DMCA takedown<\/a>&nbsp;request via Google.<\/li>\n<\/ol>\n<p>If you intentionally syndicate content to other websites, then it\u2019s worth asking them to add a canonical link to the original. That will eliminate the risk of duplicate content issues.<\/p>\n<div class=\"recommendation\"><div class=\"recommendation-title\">republishing content on your own&nbsp;site?<\/div><div class=\"recommendation-content\">\n<p>If you\u2019re republishing content from others on your site, there are two ways to prevent duplicate content issues:<\/p>\n<ol>\n<li>Canonicalize back to the original.<\/li>\n<li>Noindex the&nbsp;page.<\/li>\n<\/ol>\n<\/div><\/div>\n<h2>Final thoughts<\/h2>\n<p>Don\u2019t stress over duplicate content too much. It\u2019s usually much less of an issue than it\u2019s thought to&nbsp;be.<\/p>\n<p>If you have a handful of duplicate or near-duplicate pages, there\u2019s unlikely to be much of a problem. The same is true when quoting content from another website or other pages on your site. Small amounts of duplicate or boilerplate content should be okay. Google has systems in place to deal with such things.<\/p>\n<p>What you need to be on the lookout for are technical SEO mishaps that lead to the generation of hundreds or thousands of pages of duplicate content, such as the improper implementation of faceted navigation on ecommerce sites.<\/p>\n<p>These can wreak havoc on your crawl budget, amongst other things.<\/p>\n<p>Let me know in the comments or <a href=\"https:\/\/twitter.com\/joshuachardwick?lang=en\" target=\"_blank\" rel=\"noopener noreferrer\">on Twitter<\/a>&nbsp;if you\u2019re struggling with duplicate content.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Duplicate content is a source of constant anxiety for many site owners. Read almost anything about it, and you\u2019ll come away believing that your site is a ticking time bomb of duplicate content issues. A Google penalty is merely days&nbsp;away.<span class=\"ellipsis\">\u2026<\/span><\/p>\n<div class=\"read-more\">Read more \u203a<\/div>\n<p><!-- end of .read-more --><\/p>\n","protected":false},"author":114,"featured_media":31348,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"wp_typography_post_enhancements_disabled":false,"footnotes":""},"categories":[329],"tags":[],"coauthors":[336],"class_list":["post-31336","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technical-seo","odd"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Duplicate Content: Why It Happens and How to Fix It<\/title>\n<meta name=\"description\" content=\"Duplicate content is exact or near-duplicate content that appears on the web in more than one place. It can occur on a single website or cross-domain.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ahrefs.com\/blog\/duplicate-content\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Duplicate Content: Why It Happens and How to Fix It\" \/>\n<meta property=\"og:description\" content=\"Google says there&#039;s no such thing as a duplicate content penalty? But is this true?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ahrefs.com\/blog\/duplicate-content\/\" \/>\n<meta property=\"og:site_name\" content=\"SEO Blog by Ahrefs\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Ahrefs\/\" \/>\n<meta property=\"article:published_time\" content=\"2019-11-05T20:56:24+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-07-30T17:19:56+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/fb-duplicate-content-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"990\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Joshua Hardwick\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:description\" content=\"Google says there&#039;s no such thing as a duplicate content penalty? But is this true?\" \/>\n<meta name=\"twitter:creator\" content=\"@JoshuaCHardwick\" \/>\n<meta name=\"twitter:site\" content=\"@ahrefs\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/\"},\"author\":{\"name\":\"Joshua Hardwick\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#\\\/schema\\\/person\\\/e6a89cbde8e750d22996aa26e213e712\"},\"headline\":\"Duplicate Content: Why It Happens and How to Fix&nbsp;It\",\"datePublished\":\"2019-11-05T20:56:24+00:00\",\"dateModified\":\"2024-07-30T17:19:56+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/\"},\"wordCount\":3475,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/11\\\/fb-duplicate-content-1.png\",\"articleSection\":[\"Technical SEO\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/\",\"url\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/\",\"name\":\"Duplicate Content: Why It Happens and How to Fix It\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/11\\\/fb-duplicate-content-1.png\",\"datePublished\":\"2019-11-05T20:56:24+00:00\",\"dateModified\":\"2024-07-30T17:19:56+00:00\",\"description\":\"Duplicate content is exact or near-duplicate content that appears on the web in more than one place. It can occur on a single website or cross-domain.\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/duplicate-content\\\/#primaryimage\",\"url\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/11\\\/fb-duplicate-content-1.png\",\"contentUrl\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/11\\\/fb-duplicate-content-1.png\",\"width\":1920,\"height\":990},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/\",\"name\":\"SEO Blog by Ahrefs\",\"description\":\"Link Building Strategies &amp; SEO Tips\",\"publisher\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#organization\",\"name\":\"Ahrefs\",\"url\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/06\\\/ahrefs-logo.png\",\"contentUrl\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/06\\\/ahrefs-logo.png\",\"width\":2048,\"height\":768,\"caption\":\"Ahrefs\"},\"image\":{\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/Ahrefs\\\/\",\"https:\\\/\\\/x.com\\\/ahrefs\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/ahrefs\\\/\",\"https:\\\/\\\/www.youtube.com\\\/c\\\/ahrefscom\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/#\\\/schema\\\/person\\\/e6a89cbde8e750d22996aa26e213e712\",\"name\":\"Joshua Hardwick\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/10\\\/meme.jpg109e89523fcea81015d3cc08c79f9036\",\"url\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/10\\\/meme.jpg\",\"contentUrl\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/10\\\/meme.jpg\",\"caption\":\"Joshua Hardwick\"},\"description\":\"Head of Content @ Ahrefs (or, in plain English, I'm the guy responsible for ensuring that every blog post we publish is EPIC).\",\"sameAs\":[\"https:\\\/\\\/x.com\\\/JoshuaCHardwick\"],\"url\":\"https:\\\/\\\/ahrefs.com\\\/blog\\\/author\\\/joshua-hardwick\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Duplicate Content: Why It Happens and How to Fix It","description":"Duplicate content is exact or near-duplicate content that appears on the web in more than one place. It can occur on a single website or cross-domain.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ahrefs.com\/blog\/duplicate-content\/","og_locale":"en_US","og_type":"article","og_title":"Duplicate Content: Why It Happens and How to Fix It","og_description":"Google says there's no such thing as a duplicate content penalty? But is this true?","og_url":"https:\/\/ahrefs.com\/blog\/duplicate-content\/","og_site_name":"SEO Blog by Ahrefs","article_publisher":"https:\/\/www.facebook.com\/Ahrefs\/","article_published_time":"2019-11-05T20:56:24+00:00","article_modified_time":"2024-07-30T17:19:56+00:00","og_image":[{"width":1920,"height":990,"url":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/fb-duplicate-content-1.png","type":"image\/png"}],"author":"Joshua Hardwick","twitter_card":"summary_large_image","twitter_description":"Google says there's no such thing as a duplicate content penalty? But is this true?","twitter_creator":"@JoshuaCHardwick","twitter_site":"@ahrefs","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ahrefs.com\/blog\/duplicate-content\/#article","isPartOf":{"@id":"https:\/\/ahrefs.com\/blog\/duplicate-content\/"},"author":{"name":"Joshua Hardwick","@id":"https:\/\/ahrefs.com\/blog\/#\/schema\/person\/e6a89cbde8e750d22996aa26e213e712"},"headline":"Duplicate Content: Why It Happens and How to Fix&nbsp;It","datePublished":"2019-11-05T20:56:24+00:00","dateModified":"2024-07-30T17:19:56+00:00","mainEntityOfPage":{"@id":"https:\/\/ahrefs.com\/blog\/duplicate-content\/"},"wordCount":3475,"commentCount":0,"publisher":{"@id":"https:\/\/ahrefs.com\/blog\/#organization"},"image":{"@id":"https:\/\/ahrefs.com\/blog\/duplicate-content\/#primaryimage"},"thumbnailUrl":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/fb-duplicate-content-1.png","articleSection":["Technical SEO"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ahrefs.com\/blog\/duplicate-content\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ahrefs.com\/blog\/duplicate-content\/","url":"https:\/\/ahrefs.com\/blog\/duplicate-content\/","name":"Duplicate Content: Why It Happens and How to Fix It","isPartOf":{"@id":"https:\/\/ahrefs.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ahrefs.com\/blog\/duplicate-content\/#primaryimage"},"image":{"@id":"https:\/\/ahrefs.com\/blog\/duplicate-content\/#primaryimage"},"thumbnailUrl":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/fb-duplicate-content-1.png","datePublished":"2019-11-05T20:56:24+00:00","dateModified":"2024-07-30T17:19:56+00:00","description":"Duplicate content is exact or near-duplicate content that appears on the web in more than one place. It can occur on a single website or cross-domain.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ahrefs.com\/blog\/duplicate-content\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ahrefs.com\/blog\/duplicate-content\/#primaryimage","url":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/fb-duplicate-content-1.png","contentUrl":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/11\/fb-duplicate-content-1.png","width":1920,"height":990},{"@type":"WebSite","@id":"https:\/\/ahrefs.com\/blog\/#website","url":"https:\/\/ahrefs.com\/blog\/","name":"SEO Blog by Ahrefs","description":"Link Building Strategies &amp; SEO Tips","publisher":{"@id":"https:\/\/ahrefs.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ahrefs.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ahrefs.com\/blog\/#organization","name":"Ahrefs","url":"https:\/\/ahrefs.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ahrefs.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2023\/06\/ahrefs-logo.png","contentUrl":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2023\/06\/ahrefs-logo.png","width":2048,"height":768,"caption":"Ahrefs"},"image":{"@id":"https:\/\/ahrefs.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Ahrefs\/","https:\/\/x.com\/ahrefs","https:\/\/www.linkedin.com\/company\/ahrefs\/","https:\/\/www.youtube.com\/c\/ahrefscom"]},{"@type":"Person","@id":"https:\/\/ahrefs.com\/blog\/#\/schema\/person\/e6a89cbde8e750d22996aa26e213e712","name":"Joshua Hardwick","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/10\/meme.jpg109e89523fcea81015d3cc08c79f9036","url":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/10\/meme.jpg","contentUrl":"https:\/\/ahrefs.com\/blog\/wp-content\/uploads\/2019\/10\/meme.jpg","caption":"Joshua Hardwick"},"description":"Head of Content @ Ahrefs (or, in plain English, I'm the guy responsible for ensuring that every blog post we publish is EPIC).","sameAs":["https:\/\/x.com\/JoshuaCHardwick"],"url":"https:\/\/ahrefs.com\/blog\/author\/joshua-hardwick\/"}]}},"as_json":null,"_links":{"self":[{"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/posts\/31336","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/users\/114"}],"replies":[{"embeddable":true,"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/comments?post=31336"}],"version-history":[{"count":0,"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/posts\/31336\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/media\/31348"}],"wp:attachment":[{"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/media?parent=31336"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/categories?post=31336"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/tags?post=31336"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/ahrefs.com\/blog\/wp-json\/wp\/v2\/coauthors?post=31336"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}