Jump to content

htaccess sends bots to bogus pages


tuke

Recommended Posts

i use htaccess to rewrite from /product_info/product_id=00 to:

Product+Category/Product_Brand.html

 

the problem:

yahoo and google are indexing pages THAT DO NOT EXIST and never have.

 

they're indexing VALID category names, but they are also linking them to other categories. for instance:

 

Old+Antiques/1920s.html (valid)

the spiders find this no problem.

 

but for the past month, they have been finding url's like this:

Old+Antiques/1920s__1930s.html

what they are doing is merging one category with another, and this is producing a 200/OK response.

 

i can't figure out why.

could it be in my htaccess or my code?

 

if any part of the codes used needs to be posted.. please let me know.

 

this is from an oscommerce-based website, but i have already hit the osc forums and it has been determined to not be an oscommerce issue.

 

the htaccess and rewrite script i use is not stock oscommerce.

i have sent a bot to pull every page linked on my site, none of these url's are showing up anywhere.

 

for reference, here is my htaccess rule:

RewriteEngine on

RewriteBase /

RewriteRule ^([^/]*)\.html$ $1.php?%{QUERY_STRING} [NC]

RewriteRule ^/?(category)/([^/]*)\.html$ index.php?cPath=$2&%{QUERY_STRING} [NC]

Link to comment
https://forums.phpfreaks.com/topic/2948-htaccess-sends-bots-to-bogus-pages/
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.