PC Nerd Posted January 4, 2008 Share Posted January 4, 2008 Hi, Im looking for a way to keep my domain off google's search spiders (and preferably yahoo etc as well). Its a domain i use for presenting work and running templates trials and development tests - however its unique name causes it to appear in google when similar words are searched I know that meta tages help to categories my site and get on a spiders list - however I want to reverse that proccess. Any suggestiosn would be great. Thanks Quote Link to comment Share on other sites More sharing options...
Ken2k7 Posted January 4, 2008 Share Posted January 4, 2008 Make a robots.txt file and put this in: User-agent: * Disallow: / That's it. Since search spiders HAVE to look at the robot.txt file before looking at the site, it will read that file and you told any spider (denoted by the astericks) to not allow them to view it. Quote Link to comment Share on other sites More sharing options...
redarrow Posted January 4, 2008 Share Posted January 4, 2008 slap this on the page 1 week from now page gone.......... <head> <META NAME="ROBOTS" CONTENT="NONE"> <META HTTP-EQUIV="CACHE-CONTROL" CONTENT="NO-CACHE"> </head> and use below advise Quote Link to comment Share on other sites More sharing options...
PC Nerd Posted January 4, 2008 Author Share Posted January 4, 2008 u, - those meta tags - do those need to be on all my pages? thanks edit:: do i have to place robots.txt in the foot - or in every directory - and with that can i place different robots.txt fiels in different directories so that it will change per directory?? if a /members/ wont have robot access thnks Quote Link to comment Share on other sites More sharing options...
Ken2k7 Posted January 4, 2008 Share Posted January 4, 2008 do i have to place robots.txt in the foot - or in every directory - and with that can i place different robots.txt fiels in different directories so that it will change per directory?? if a /members/ wont have robot access thnks robots.txt HAS to go in the ROOT directory. Put it somewhere else and it'll fail. So to your second question, no, you can't. But you can allow it robots to certain directories using something like this: User-agent: * Disallow: /images/ Disallow: /members/ That would allow robots everywhere EXCEPT in the /images/ and /members/ directory. Hope that helps. Quote Link to comment Share on other sites More sharing options...
PC Nerd Posted January 4, 2008 Author Share Posted January 4, 2008 thankx - ill assume thats the web accessable root - im still confused betweenthe two. (/www not /) Thanks for all your help Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.