Search the Community
Showing results for tags 'googlebot being blocked'.
Found 1 result
-
My Rewrite condition doesn't work (htaccess)? I'm trying to allow only the bots bellow to access my site content, however, I'm receiving 403 error every time I try to submit a sitemap to Google webmaster tools. When I delete the rules, I stop getting the error. It's more like Googlebot is the wrong name of the bot.. IDK... What I need to do is to allow Googlebot, bing, yahoo and altavista, then block any other bot that tries to access the website. My rules are as follows: RewriteEngine on RewriteBase / -----> There are other rules here, just system related. <----- RewriteCond %{HTTP_USER_AGENT} Googlebot [OR] RewriteCond %{HTTP_USER_AGENT} Googlebot-2.1 [OR] RewriteCond %{HTTP_USER_AGENT} Googlebot-Mozilla-2.1 [OR] RewriteCond %{HTTP_USER_AGENT} Google-AdSense-2.1 [OR] RewriteCond %{HTTP_USER_AGENT} Googlebot-Image [OR] RewriteCond %{HTTP_USER_AGENT} AdsBot-Google [OR] RewriteCond %{HTTP_USER_AGENT} msnbot [OR] RewriteCond %{HTTP_USER_AGENT} AltaVista [OR] RewriteCond %{HTTP_USER_AGENT} Yahoo-Slurp [OR] RewriteCond %{HTTP_USER_AGENT} Alexa-1 [OR] RewriteCond %{HTTP_USER_AGENT} Alexa-2 [OR] RewriteCond %{HTTP_USER_AGENT} MSN-1.0 [OR] RewriteCond %{HTTP_USER_AGENT} Slurp RewriteRule .* - [F,L]
