liamloveslearning Posted January 14, 2010 Share Posted January 14, 2010 Hi everyone, I have my robots.txt file in my root foler, my robots.txt is User-agent: * Allow: / User-agent: * Disallow: /eshot/ User-agent: * Disallow: /newsletters/ User-agent: * Disallow: /Connections/ User-agent: * Disallow: /libraries/ User-agent: * Disallow: /resources/ User-agent: * Disallow: /sifr/ User-agent: * Disallow: /SpryAssets/ in webmaster tools on google I receive the message "www.mywebiste.com Allowed by line 2: Allow: / Detected as a directory; specific files may have different restrictions" Does this mean where I have my allow all, that its contradicting my disallow restrictions? Link to comment https://forums.phpfreaks.com/topic/188448-robotstxt/ Share on other sites More sharing options...
AngelG107 Posted January 14, 2010 Share Posted January 14, 2010 I took the freedom to edit your robots.txt file, try this : User-agent: * Disallow: /eshot/ Disallow: /newsletters/ Disallow: /Connections/ Disallow: /libraries/ Disallow: /resources/ Disallow: /sifr/ Disallow: /SpryAssets/ That's basically is going to do all you're looking for, it will disallow those directories in all search engines, Link to comment https://forums.phpfreaks.com/topic/188448-robotstxt/#findComment-995054 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.