Duplicate Content In WordPress – Begone!

Duplicate content in wordpress? I cast thee from my site!

So where does it come from?

Well, the first source of duplicate content will come from you not putting the proper items in your robots.txt.

Here’s what my robots.txt file looks like:

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-content/plugins/wp-chunk.php
    Disallow: /wp-content/plugins/wp-db-backup.php
    Disallow: /wp-content/plugins/st-orderpages.php
    Disallow: /wp-content/plugins/pollpress/
    Disallow: /wp-content/plugins/buddyCards/
    Disallow: /wp-content/plugins/akismet/
    Disallow: /wp-content/plugins/advanced-wysiwg.php
    Disallow: /wp-content/plugins/sociable/
    Disallow: /wp-content/plugins/custom-smileys.php
    Disallow: /wp-content/plugins/autometa/
    Disallow: /wp-content/plugins/lmbbox-smileys/
    Disallow: /wp-content/plugins/wodspewm/
    Disallow: /wp-content/themes/
    Disallow: /wp-includes/
    Disallow: /category/
    Disallow: /page/
    Disallow: /feed/
    Disallow: /comments/

You’ll notice that I stopped the bots from crawling in all of my plugins folder. I could have simply used Disallow: /wp-content/ – unfortunately, doing so would have led to none of the translated pages getting indexed since the translation icons at the bottom of this page come from a plugin that resides in /wp-content/plugins/global-translator.

Anyhow, make sure you don’t block anything that you are using inside of your pages with direct links to it like I mentioned and you’ll be fine. Then you’ll block the rest of the includes, category, feed, page, and comments folders.

It’s all precisely designed to keep the duplicate content from your pages!

The second source of duplicate content from wordpress is archive pages. This one was a bit trickier to solve. To do so I had to modify my wordpress template. This isn’t for the faint of heart!

The code to do what I’ve done on my archive pages where it only lists the title looks like this:

Posted in

By making these changes you remove the duplicate content from your archive pages because they’ll no longer have all of the articles for that month on the page. It’ll only be on the single page like it should be!

G-Man

P.S. Free link to anyone who can tell me how to close up the gap on those links on my archives page so that it doesn’t look like it’s double-spaced :)

Leave a Reply