droidus Posted July 17, 2011 Share Posted July 17, 2011 how do i add a file to a directory that prevents all bots from accessing a directory? Quote Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/ Share on other sites More sharing options...
TheSky Posted July 17, 2011 Share Posted July 17, 2011 User-agent: * Disallow: /folder/ and save as robots.txt in your root folder Quote Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243854 Share on other sites More sharing options...
droidus Posted July 17, 2011 Author Share Posted July 17, 2011 so with that /folder/, i would just insert the path to my directory that i want to block, correct? Quote Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243855 Share on other sites More sharing options...
TheSky Posted July 17, 2011 Share Posted July 17, 2011 yes Quote Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243858 Share on other sites More sharing options...
ignace Posted July 17, 2011 Share Posted July 17, 2011 A robots.txt doesn't prohibit them from scanning it anyway. Can you elaborate what you want to achieve? If no one may look into the directory you can use a .htaccess to disable directory scanning using: Options -Indexes This is the recommended method. Quote Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243886 Share on other sites More sharing options...
droidus Posted July 17, 2011 Author Share Posted July 17, 2011 so i create a .htaccess in my directory that i don't want the bots to see (but only people when they have the link), and then type in it "Options -Indexes"? Quote Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243913 Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.