Showing 404 Page based on URL Structure

[natasha_thomas]

Friends,

 

I hope all had great Vacations. 

 

Have a look at this URL:

 

URL1: www.mysite.co.uk/a.html

URL2: www.mysite.co.uk/air-purifier.html

 

Am not sure how, but google has indexed so many URLs like URL1 for my site, where the "a" can be b, c, 1 , 3, 5, ed, wq so on..... not sure how google got them.

 

URL2, is the ideal URL i am looking for.

 

So, what i want is, i want to show a 404 Page when the URL structure looks like URL1 coz a.html make no sense unlike air-purifier.html.

 

So the logic is, if there are lesser than 4 Letters between "www.mysite.co.uk/" and ".html", i want to show a 404 page.

 

I think it needs to be done at .htaccess level. No?

 

May someone help me with the code needed to achieve this?

 

 

Cheers

Natash Thomas

[lastkarrde]

I'm pretty sure you can remove your pages through Google's webmaster tools (more http://www.google.com/support/webmasters/bin/answer.py?answer=164734). If the pages do not exist Google should automatically remove them within a couple of months.

[gizmola]

Hi Natash,

  If a.html is not a valid page, then the webserver will already issue a 404.  I'm guessing here that these pages might actually point to content?  If so, then what you want to do is have them issue a 301, which is read as "permanently moved."  Google looks at this and honors it, so you don't lose people who basically get a big error page.

 

If you want a customized 404 page you can implement this in your htaccess with:

 

ErrorDocument 404 /error404.php

 

And of course you will need to make the error404.php file and have it look and say what you want it to.

 

[natasha_thomas]

Ops i made a mistek here. I forgot to mention one thing.

 

Mine is a dynamic script which generates content for whatever keyword you pass in the URL.

 

So in a.html "a" is treated as a keyword.

 

So the script will still generate content for this junk keyword.

 

Which is why, i want a .htaccess level solution, so for any dynamic keyword which is lesser than 4 characters lenght, should get a 404 Header response.

 

Any solution for this?

 

[JonnoTheDev]

You will need to edit the Mod-Rewrite rule in your .htaccess

 

An example of this:

# match the characters a-z between 4 and unlimited times
[a-z]{4,}

[natasha_thomas]

You will need to edit the Mod-Rewrite rule in your .htaccess

 

An example of this:

# match the characters a-z between 4 and unlimited times
[a-z]{4,}

 

Thanks NJ,

 

Just wondering where are we controlling that we need ot look between domain name and .html?

 

eg: www.domains.com/abc.html

so we need to check the no of letters between www.domain.com/ and .html which is abc.

 

Where are we handlling this in htaccess with [a-z]{4,}?

 

Cheers

[JonnoTheDev]

Post the contents of your .htaccess file

[natasha_thomas]

NJ,

 

Here is my HTACCESS:

 

<IfModule mod_rewrite.c>

   Options +FollowSymLinks
   RewriteEngine On
RewriteRule ^([^\\\/.]*)\.htm $1.html
   # If you experience issues with SEO urls
   # not working as expected, try removing
   # the # below to enable RewriteBase

   #RewriteBase /

RewriteRule ^images/e/(.*)$ http://thumbs.ebaystatic.com/pict/$1 [R,L]
RewriteRule ^item-(.*)_(.*)_(.*)_(.*).html$ auction.php?title=$1&item=$2&country=$3&ccid=$4
RewriteRule ^item-(.*)_(.*)_(.*).html$ auction.php?title=$1&item=$2&country=$3
RewriteRule ^item-(.*)_(.*).html$ auction.php?title=$1&item=$2
   

   RewriteCond %{SCRIPT_FILENAME} -f [OR]
   RewriteCond %{SCRIPT_FILENAME} -d [OR]
   RewriteCond %{ENV:REDIRECT_STATUS} !^$


   RewriteRule .* - [L]

   



   RewriteRule sitemap.xml sitemap.php
   RewriteRule ^browse/?$ browse.php [L]



# Please don't forget to change extension name in this file for all: php$ to html$ or html$ to php$ , etc.
# If you have changed ones in admin area -> Store Configuration -> SEO Options-> SEO Settings  (Extension for web pages)

  #####################
  # Product URL Rules #
  #####################

   #RewriteRule ^p/([^/]*)/([^/]*)/([^/]*)/([^/]*)/([0-9]+)\.html$ product.php?p1=$1&p2=$2&p3=$3&p4=$4&p=$5 [QSA,L]
   RewriteRule ^p/([^/]*)/([^/]*)/([^/]*)/([^/]*)\.html$ product.php?p1=$1&p2=$2&p3=$3&p4=$4 [QSA,L]
   RewriteRule ^p/([^/]*)/([^/]*)/([^/]*)/([0-9]+)\.html$ product.php?p1=$1&p2=$2&p3=$3&p=$4 [QSA,L]
   RewriteRule ^p/([^/]*)/([^/]*)/([^/]*)\.html$ product.php?p1=$1&p2=$2&p3=$3 [QSA,L]
   RewriteRule ^p/([^/]*)/([^/]*)/([0-9]+)\.html$ product.php?p1=$1&p2=$2&p=$3 [QSA,L]
   RewriteRule ^p/([^/]*)/([^/]*)\.html$ product.php?p1=$1&p2=$2 [QSA,L]

  ######################
  # Category URL Rules #
  ######################

   RewriteRule ^(.*)/([0-9]*)/(.*)/feed/?$ rss.php?c=$1&p=$2&sort=$3 [QSA,L] # Category URL Rule
   RewriteRule ^(.*)/([0-9]*)/(.*)?$ browse.php?c=$1&p=$2&sort=$3 [QSA,L] # Category URL Rule
   RewriteRule ^(.*)/([0-9]*)?$ browse.php?c=$1&p=$2 [QSA,L] # Category URL Rule
  
  ######################
  # Static Page URL Rules #
  ######################
   
   RewriteRule ^content/([^/]*)\.php$ pages.php?p=$1 [QSA,L]
  
   ######################
  # Search Page URL Rules #
  ######################

   RewriteRule ^(.*)\.html$ search.php?q=$1 [QSA,L]
   
</IfModule>

[gizmola]

It looks like these fall through to the final RewriteRule:

 

RewriteRule ^(.*)\.html$ search.php?q=$1 [QSA,L]

 

So if you use neils suggestion then change that to:

 

RewriteRule ^[a-z]{4,}\.html$ search.php?q=$1 [QSA,L]

 

This rule very specifically will only allow lowercase letters, with no numbers or other characters in there, just so you know. 

[natasha_thomas]

It looks like these fall through to the final RewriteRule:

 

RewriteRule ^(.*)\.html$ search.php?q=$1 [QSA,L]

 

So if you use neils suggestion then change that to:

 

RewriteRule ^[a-z]{4,}\.html$ search.php?q=$1 [QSA,L]

 

This rule very specifically will only allow lowercase letters, with no numbers or other characters in there, just so you know.

 

GM, 

 

Thank you so Much...

 

But what i rather wanted was, 404 where the number of letters are lesser than 4.

 

But as you rightly said, [a-z]{4,} is not working for numbers or even " - "  not even Upper Cases.  :'(

 

What code change do we need to achieve this?

 

will it be like:  [a-zA-Z0-9-]{4,}    or something like that?

 

 

 

 

[gizmola]

Yes you got it.  My assumption was that if none of these rules match, then you will fall through to a 404 automatically.

[natasha_thomas]

Yes you got it.  My assumption was that if none of these rules match, then you will fall through to a 404 automatically.

 

Ok here is what i am using:

 

RewriteRule ^[a-zA-Z0-9-]{4,}\.html$ search.php?q=$1 [QSA,L]

 

But, after this change, its not passing the value of variable "q" .

 

What am i doing wrong?

 

 

 

 

[gizmola]

Oops, yes you need parens around the part you want to capture to the $1 variable.

 

RewriteRule ^([a-zA-Z0-9-]{4,})\.html$ search.php?q=$1 [QSA,L]

[natasha_thomas]

Oops, yes you need parens around the part you want to capture to the $1 variable.

 

RewriteRule ^([a-zA-Z0-9-]{4,})\.html$ search.php?q=$1 [QSA,L]

 

It was Great Help Gizmola.

 

One last question: I rather want to show a custom 404.php page and not that default page.

 

so on 404.php i can even write few things and ask visitor to click on categoris.

 

How can we achieve this?

[gizmola]

Yes, it's just like any other php file --- you can have markup, queries, php code, etc.  Most people make it look just like a normal page on their site, and in the content area, you would typically have some text saying that the Page was not found, and typically suggesting that people go somewhere else.

[natasha_thomas]

Yes, it's just like any other php file --- you can have markup, queries, php code, etc.  Most people make it look just like a normal page on their site, and in the content area, you would typically have some text saying that the Page was not found, and typically suggesting that people go somewhere else.

 

Yes i get that bit but how to implement it? Do i need ot create a file 404.php in my root folder and what change i need to make in htacess to redirect all 404 to 404.php?

 

Cheers

[JonnoTheDev]

As Gizmola stated add this to the bottom of your .htaccess document

 

ErrorDocument 404 /404.php

 

Now you can create the 404.php file in your document root and this page will be served whenever a page is not found.

[natasha_thomas]

want to Thank Gizmola and Niel for Guidance and Support.

 

Many Many Thanks

Natasha