Mutley Posted March 18, 2009 Share Posted March 18, 2009 Is there a rewrite to stop spiders looking through pages under a certain directory? Thanks in advance, Nick. Quote Link to comment https://forums.phpfreaks.com/topic/150024-search-spiders/ Share on other sites More sharing options...
corbin Posted March 18, 2009 Share Posted March 18, 2009 You could use a robots.txt. Quote Link to comment https://forums.phpfreaks.com/topic/150024-search-spiders/#findComment-788013 Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.