droidus Posted July 17, 2011 Share Posted July 17, 2011 how do i add a file to a directory that prevents all bots from accessing a directory? Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/ Share on other sites More sharing options...
TheSky Posted July 17, 2011 Share Posted July 17, 2011 User-agent: * Disallow: /folder/ and save as robots.txt in your root folder Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243854 Share on other sites More sharing options...
droidus Posted July 17, 2011 Author Share Posted July 17, 2011 so with that /folder/, i would just insert the path to my directory that i want to block, correct? Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243855 Share on other sites More sharing options...
TheSky Posted July 17, 2011 Share Posted July 17, 2011 yes Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243858 Share on other sites More sharing options...
ignace Posted July 17, 2011 Share Posted July 17, 2011 A robots.txt doesn't prohibit them from scanning it anyway. Can you elaborate what you want to achieve? If no one may look into the directory you can use a .htaccess to disable directory scanning using: Options -Indexes This is the recommended method. Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243886 Share on other sites More sharing options...
droidus Posted July 17, 2011 Author Share Posted July 17, 2011 so i create a .htaccess in my directory that i don't want the bots to see (but only people when they have the link), and then type in it "Options -Indexes"? Link to comment https://forums.phpfreaks.com/topic/242209-bots-block-from-scanning-directory/#findComment-1243913 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.