Jump to content

Log problem, 80% storage usage overnight resulting in full server.


greenace92

Recommended Posts

I rent this 10 GB VPS and I am using 1.5 GB or less.

 

This is the first time this has happened to me (I hope the last).

 

I'm not sure what caused the problem but one day I could not log into PHPMyAdmin, after searching it turned out that it was because the server was full.

 

Maybe it wasn't over night, maybe I just hadn't seen the storage in a while. Although I would have noticed on my server web panel at the hosting company's website.

 

There was something about rotation of server logs. Disabling mysqld general logging.

After deleting a lot of files, I still could not get rid of the 8 GB chunk... I didn't know where it was so I backed up my files/mysql databases and re-installed the server.

 

I'd like to know how I could prevent this in the future.

Any thoughts?

Link to comment
Share on other sites

Not without knowing where the space went. If it happens again, use (Linux?) df to see how much space various directories are taking. And you can always ask your hosting provider for support - at least with identifying where the problem is, if not also fixing it.

Link to comment
Share on other sites

Not without knowing where the space went. If it happens again, use (Linux?) df to see how much space various directories are taking. And you can always ask your hosting provider for support - at least with identifying where the problem is, if not also fixing it.

 

Hey requinix,

 

I immediately contacted my hosting provider but they didn't really have any answers. I was panicking thinking "Oh my God was I hacked?" They weren't really much of help.

 

When this happened I had posted on webhosting talk and I was told about df -h,  du -sh /*, and du -sm /directory

 

It appeared that /var/log had the highest at 8.3G (not shown above), just looking at my previous thoughts.

 

This one file alone is already 1.1G in size

 

error.log.1 under /var/log/apache2

 

It was suggested to set up the logrotate to catch 100M intervals and every hour. I'll look into that, I saw that it was already enabled.

Edited by greenace92
Link to comment
Share on other sites

 

Can you confirm that logrotate is running? Check its logs to see when it last ran.

 

I believe it is running because I just removed 8 zipped files which the file type gz was appended to the old logs by incremented numbers eg. access.log.2.gz

 

Well there was a duplicate problem, I removed the initial script

 

After implementing the script below from a tutorial (having changed directories to /var/log/apache2/access.log or error.log)

When I try to manually call the logrotate by /etc/cron.daily/logrotate, nothing seems to happen. I check the directory and there aren't any new compressed files. hmm, I think I'll try the original and see what happens.

/usr/local/apache2/logs/access_log /usr/local/apache2/logs/error_log {
    size 100M
    compress
    dateext
    maxage 30
    postrotate
      /usr/bin/killall -HUP httpd
      ls -ltr /usr/local/apache2/logs | mail -s "$HOSTNAME: Apache restarted and log files rotated" ramesh@thegeekstuff.com
    endscript
}
Edited by greenace92
Link to comment
Share on other sites

That latest conf file won't rotate stuff immediately. Only if the file is at least 100MB.

 

The rotate won't happen as soon as the file hits the limit. It's just a precondition to rotating. Could it be that there just were a ton of errors within the rotation timeframe?

Link to comment
Share on other sites

That latest conf file won't rotate stuff immediately. Only if the file is at least 100MB.

 

The rotate won't happen as soon as the file hits the limit. It's just a precondition to rotating. Could it be that there just were a ton of errors within the rotation timeframe?

 

Well error.log.1 is 656M so, I'm not sure why it isn't triggering, maybe I missed something...

 

Maybe I have to set the top directories/files to have the .1 extension to zip that one, then by default the only one to be zipped is the regular error.log maybe...

 

maybe not

 

error running non-shared postrotate script for ...

Edited by greenace92
Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.