oceans Posted July 21, 2007 Share Posted July 21, 2007 Dear Friends, I am setting up my web site, though it is still under construction. I get the following error message being recorded in my error log file. " [client XX.6.75.16] File does not exist: robots.txt " I agree I do not have this file, but I do not understand why am I getting this. Can any one help me, must I put this file in my root? Link to comment https://forums.phpfreaks.com/topic/61077-solved-robotstxt/ Share on other sites More sharing options...
hackerkts Posted July 21, 2007 Share Posted July 21, 2007 Do you know what's robots.txt for? If you're not sure then just search about it, anyway putting it will not harm you. robots.txt User-Agent: * Allow: Link to comment https://forums.phpfreaks.com/topic/61077-solved-robotstxt/#findComment-304027 Share on other sites More sharing options...
oceans Posted July 23, 2007 Author Share Posted July 23, 2007 Thanks, I just read on on the web, thanks, I understood. Link to comment https://forums.phpfreaks.com/topic/61077-solved-robotstxt/#findComment-305038 Share on other sites More sharing options...
hackerkts Posted July 23, 2007 Share Posted July 23, 2007 You're welcome! Link to comment https://forums.phpfreaks.com/topic/61077-solved-robotstxt/#findComment-305191 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.