Duplicate content and seo?

[galvin]

If you host a website at abusiness.com that has been live for years and you are now resigning the website at another domain (like webcompany.com/redesigns/abusiness), how can you avoid being penalized by Google for duplicate content?

 

In other words, I will be copying all of the content from the live site to this new, in-development site, so all content will be exactly the same (I will just be changing the design)

 

So while the new site is in development, there will be two websites at two different domains with the exact same content, which I'm sure will piss Google off

 

In case you're wondering, the in-development site has to be live on the internet so the client can keep reviewing the progress (from another state).

 

Any thoughts?

[Pikachu2000]

You could require a username/password.

[galvin]

Good thought.  Would using htaccess to only allow visits from my computer and the client's computer also work? (see example below).  In other words, would that be enough to keep Google from finding the in-development site? 

 

order deny,allow
deny from all
allow from 173.251.xx.x
allow from 24.187.xxx.xxx

[Pikachu2000]

It should, but if your client's IP address changes or he wants to show someone the site on his mobile device, what happens? As far as I know, googlebot respects robots.txt file directives, so you could do that as well.

[galvin]

Awesome. I just googled based on your reply and found the following.  This sounds like a good solution, but does anyone know for sure if this will accomplish my goal (i.e. simply to prevent being penalized for duplicate content)...

 

So, if you want to effectively hide pages from the search engines, and this might seem contradictory, you need them to index those pages. Why? Because when they index those pages, you can tell them not to List them. There's two ways of doing that: by using robots meta tags, like this (and I've got an article on robots meta tags that's more extensive):

 

<meta name="robots" content="noindex,nofollow"/>

 

The issue with a tag like that is that you have to add it to each and every page. That's why the search engines came up with the X-Robots-Tag HTTP header. This allows you to specify an HTTP header called X-Robots-Tag, and set the value as you would the meta robots tags value. The cool thing about this is that you can do it for an entire site. So, if your site is running on Apache, and mod_headers is enabled (it usually is), you could add the following single line to your .htaccess file:

 

Header set X-Robots-Tag "noindex, nofollow"

 

And it'd have the effect that that entire site can be indexed, but will never be shown in the search results. So, get rid of that robots.txt file with Disallow: / in it, and use the X-Robots-Tag instead!

[floridaflatlander]

I don't know if you'll be penalized or not, I think the penalty comes from duplicate information on the same site.

 

People publish and republish news all the time on different web sites. How does Google know or not know if they are related?

 

You can also put a no-follow on your new site to prevent confusion, when the time comes use a 301 redirect and remove the no follow. Make the file paths a logical path that will help you easily implement your 301's. It will take 2-4 weeks for almost all the search engines to drop your old website name and start listing the new one. I have had one photograph that was at nine weeks before it was redirected(on bing). Until then, search engines will list old URLs and when someone clicks them they will be automatically redirected to the new site.

[galvin]

Interesting thought.  Maybe there would be no penalty.  Because I was thinking that if it was a penalty, I could copy all content from a competitors website and post it at my own domain and try to hurt them.  So it would make sense if that wasn't actually a penalty. 

 

Anyone know for sure, this is an interesting topic (to me at least)

 

[floridaflatlander]

I could copy all content from a competitors website and post it at my own domain and try to hurt them.

 

? ... I hope that was a joke.

[galvin]

It wasn't a joke, just a "what if."  I'm not going to do that, I was just saying that people could easily do that and because it's so easy, that makes me think that having duplicate content on TWO different domains will NOT draw a penalty from Google.