maximum number of internal redirects problem???

[sokrates4612]

Hello,
I don´t know if this is a problem of PHP, or Apache, or Mysql.
After few hours, my virtual server hosting reaches very high processes number, after that it is impossible to connect to any webservice, please see attachement. Only a reboot helps.

In Apache (in /var/log/apache2/error_log) I see a strange records:

[Sun Dec 10 18:35:40 2006] [error] [client 91.127.179.50] Directory index forbid
den by rule: /data/vsevedko/html/_sub/recepty/styles/, referer: http://recepty.v
sevedko.sk/searchform.php
[Sun Dec 10 18:36:11 2006] [error] [client 91.127.179.50] mod_rewrite: maximum n
umber of internal redirects reached. Assuming configuration error. Use 'RewriteO
ptions MaxRedirects' to increase the limit if neccessary.
[Sun Dec 10 18:39:36 2006] [error] [client 74.6.85.168] mod_rewrite: maximum num
ber of internal redirects reached. Assuming configuration error. Use 'RewriteOpt
ions MaxRedirects' to increase the limit if neccessary.
[Sun Dec 10 18:42:45 2006] [error] [client 84.47.94.154] Directory index forbidd
en by rule: /data/vsevedko/html/_sub/recepty/styles/, referer: http://recepty.vs
evedko.sk/submitrecipe.php
...

I checked .htaccess in /data/vsevedko/html/_sub/recepty/styles/ and all around, regarding to http://www.sitepoint.com/print/mod_rewrite-no-endless-loops
but there is only Options -indexes set.

What should I do to prevent server flood?

Thanks for your help.



[attachment deleted by admin]

[steviewdr]

Humm - thats very very bad. Apache2 should know better.

I suggest you look into this one errror:
"mod_rewrite: maximum num
ber of internal redirects reached. Assuming configuration error. Use 'RewriteOpt
ions MaxRedirects' to increase the limit if neccessary."

Do you use mod_rewrite? I would disable it and see. Otherwise look for all rewrite configs you have and check them.
Also - you need to narrow down this a little more:
Is it 1 or 2 accessing servers causing this? Netstat will show connections etc.
Is it a gradual build up of connections over time?
Start shutting down areas of your website trying to debug where the problem is.
Id say its apache as fault. A configuration in a htacess file more than likely.

-steve

[sokrates4612]

Well, if you said it is bad, I think it is really very bad.  :'(

Do I really have to regard and check all sites on vserver? Or the only one mentioned in the error log???

If I had to turn off mod_rewrite - I only could close my job, because all of my sites uses it.

Using netstat: can you please give me some hints how and when use it for correct problem debuging?

[steviewdr]

[quote]Do I really have to regard and check all sites on vserver? Or the only one mentioned in the error log???[/quote]
You have to check all htaccess and apache config files IMO.
There may be other error logs also.

[quote]If I had to turn off mod_rewrite - I only could close my job, because all of my sites uses it.[/quote]
If you are using that much mod_rewrite - there might be issues there. Post a sample of the rewrite rules you use.

[quote]Using netstat: can you please give me some hints how and when use it for correct problem debuging?[/quote]
Well using netstat along with other tools will show the number of active connections etc.
I also suggest you install munin on your server. It will give a complete report of whats going on.

-steve

[sokrates4612]

Rewrite rules I use:
[code]RewriteEngine On

RewriteRule ^detail/(.*)/(.*).html detail.php?linkid=$1 [L,NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-l
RewriteRule ^.* browse.php [L,NC][/code]

and also:
[code]RewriteEngine On

RewriteCond %{HTTP_HOST} ^inzercia66\.sk [nc]
RewriteRule (.*) http://www.inzercia66.sk/$1 [R=301,L]

# Show ad
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/posts/([0-9]+)(_[^/]*)?/([0-9]+)(_[^/]*)?/([0-9]+)(_[^/]*)?\.html index.php?view=showad&adid=$7&cityid=$1 [QSA]

# Ads
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/posts/([0-9]+)(_[^/]*)?/([0-9]+)(_[^/]*)?/page([0-9]*)\.html index.php?view=ads&catid=$3&subcatid=$5&cityid=$1&page=$7 [QSA]
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/posts/([0-9]+)(_[^/]*)?/([0-9]+)(_[^/]*)? index.php?view=ads&catid=$3&subcatid=$5&cityid=$1 [QSA]

# Ads by cat
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/posts/([0-9]+)(_[^/]*)? index.php?view=ads&catid=$3&cityid=$1 [QSA]

# Show event
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/events(/(([0-9]+)-([0-9]+)-([0-9]+)))?/([0-9]+)(_[^/]*)?\.html index.php?view=showevent&date=$4&adid=$8&cityid=$1 [QSA]

# Events on date
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/events/(([0-9]+)-([0-9]+)-([0-9]+))/page([0-9]*)\.html index.php?view=events&date=$3&cityid=$1&page=$7 [QSA]
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/events/(([0-9]+)-([0-9]+)-([0-9]+)) index.php?view=events&date=$3&cityid=$1 [QSA]
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/events index.php?view=events&cityid=$1 [QSA]

# Show image
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/images/([^/]+)/([0-9]+)(_[^/]*)?\.html index.php?view=showimg&posterenc=$3&imgid=$4&cityid=$1 [QSA]

# Images by user, paged
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/images/([^/]+)/page([0-9]+)\.html index.php?view=imgs&posterenc=$3&cityid=$1&page=$4 [QSA]

# Top Images, paged
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/images/page([0-9]+)\.html index.php?view=imgs&cityid=$1&page=$3 [QSA]

# Images by user
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/images/([^/]+) index.php?view=imgs&posterenc=$3&cityid=$1 [QSA]

# Top Images
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/images index.php?view=imgs&cityid=$1 [QSA]

# Quick ad/event/image
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/showpost/([0-9]+)(_[^/]*)?\.html index.php?view=showad&adid=$3&cityid=$1 [QSA]
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/showevent/([0-9]+)(_[^/]*)?\.html index.php?view=showevent&adid=$3&cityid=$1 [QSA]
RewriteRule ^inzercia/([-0-9]+)(_[^/]*)?/showimage/([0-9]+)(_[^/]*)?\.html index.php?view=showimg&imgid=$3&cityid=$1 [QSA]


# City/Region
RewriteRule ^inzercia/([-0-9]+) index.php?view=main&cityid=$1 [QSA]
RewriteRule ^inzercia/([-0-9]+) index.php?view=main&cityid=$1 [QSA]
[/code]


Before a few minutes it also happens - I had to reboot vserver. But before rebooting I tried netstat with, and without "-a" attribute. Please see files attached.

If you have some ideas, what should I do, please help me. Thank you very very very much.

[attachment deleted by admin]

[steviewdr]

Hummm - definately very bad.

I think its all the "SYN_RECV" causing the problem. You can try and adjust apache settings such as:
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 1
<IfModule prefork.c>
StartServers        5
MinSpareServers      5
MaxSpareServers    10
MaxClients          20
MaxRequestsPerChild  0
</IfModule>

<IfModule worker.c>
StartServers        2
MaxClients        150
MinSpareThreads    25
MaxSpareThreads    75
ThreadsPerChild    25
MaxRequestsPerChild  0
</IfModule>


<IfModule perchild.c>
NumServers          5
StartThreads        5
MinSpareThreads      5
MaxSpareThreads    10
MaxThreadsPerChild  20
MaxRequestsPerChild  0
AcceptMutex fcntl
</IfModule>


I dont have time to go into it at the moment. But you need to configure apache2 somewhat.

As for the rewrites - there is quite a lot of them ;-) !!

When did this start happenning? Was something added or configed or changed on the server?

Rgds,
Steve

[sokrates4612]

Thanks. I made those changes, but almost everything was configured exactly as you wrote, except KeepAliveTimeout (before 15) Maxclients in prefork.c (before 150).

But will my sites now work correctly even in a traffic peak? I have 4 site hosted on this vserver and it is about 5000 pageviews/day. Will have these changes some negative effect to a vserver performance in term of common visitors??

Apropos rewrites: is it problem for vserver using many rewrite rules? I need them all to make "nice URLs". :-)

[code]When did this start happenning? Was something added or configed or changed on the server?[/code]

It´s strange, but not. I only had some problems with database encoding, but I didn´t any huge changes...

[steviewdr]

I didnt mean for you to just copy them straight in - but to research them settings.

Also I had made a typo when I copied in my example settings:
KeepAliveTimeout 1 is meant to be [b]KeepAliveTimeout 15 [/b] (as you had it!) Apologies - I didnt realise you were going to copy and paste it in.
Those settings I copied in - are Debian sarge's default ones.

As I said - I suggest you look up more on setting these values.
Specifically I think you had too many "SYN_RECV" in your netstat and you need to do some looking up on this.

Also - as another note: I had problems before with my MySQL server running out of max connections, and as a result the webserver sufferred. I increased the max_connections in MySQL and it somewhat helped (albeit there were other factors and causes).

Bottom line - go read up on "tweaking apache2".

-steve

[sokrates4612]

Ok,
I changed 1 to 15 :-)

So you mean that I should experiment with those settings??? Uff, it could be a hard work, there are so much combinations...

Maybe it will be a MySQL problem, because, now I got a message like:
Warning: mysql_connect() [function.mysql-connect]: Too many connections in...

Thanks for your help.
Does somewhere exists some paid support for apache/php/mysql/linux administration? :-)

[steviewdr]

[quote]Maybe it will be a MySQL problem, because, now I got a message like:
Warning: mysql_connect() [function.mysql-connect]: Too many connections in...[/quote]

FACT! That could well be your problem!! I had the very same.

vi /etc/mysq/my.cnf

//add in this line:
set-variable = max_connections=250

The default is 150 connections, and depending on how long your php scripts (etc.) run for they may be keeping the mysql connections open and slowing the server down.

As for my background, I am one of the voluntary root dmins for the computer society (www.skynet.ie) in the university here. I dont have the time for providing full time support atm however.

See if the above mysql config entry helps.
-steve

[sokrates4612]

Well, I changed it to 300 :-) and there was no problem (from monday) until today (to saturday)!
Now the same thing happes, I had to restart vserver...
I suspect that this problem is caused by some searchbot, maybe from yahoo, because it happens when it was visiting my website. I think I try also increase RewriteOptions Maxredirects.

[steviewdr]

Limit the searchbots to some parts of the websites only. I know you can set the googlebot how often to re-spider your website.

Its definately some config or setting that needs tweaking. Keep a watchful eye on the logs.

-steve

[sokrates4612]

Uff, before few minutes everything repeats. But I was "in the middle of it".

I didnt had to restart whole vserver, I´ve just stopped apache2 and then turned it on again. And it helps.

But I think there is no problem with Mysql, but with Apache2 config. Now I am decreasing Maxclients to 100 (before 150) in worker and prefork module. I also decreased ListenBacklog to 10 (default 150?).

I wonder if this can flick away search bots - that is not what I want! I only want my vserver to breath, when they crawl my sites...  ;D

If I had a better serverhosting parameters, and not only virtual serverhosting (500 Mhz, 256 RAM, 1GB SWAP...), could it solve my problems absolutely?

Does it work this way? - the more pages for viewing on your server you have, the more server performance you need, when a spy bots diggs into you websites? So server performance is mainly for bots, not for visitors...?  ::)