Help me understand why PHP is messing with my RAM!

[B0b]

Hi guys,

 

I'm having an headache at trying to understand what's wrong with PHP in memory freeing.

 

Basically, I got a simple script transforming a massive file (60mb) to an array using file() function.

Everything goes fine, but the RAM used is never freed! Even if I use unset on the array (which is the only var of the script).

If I use memory_get_usage, everything is normal, but it's the RAM usage in my server's Plesk control panel that always keep growing up and never lower.

 

I am using "ini_set('memory_limit', '500M' );" at the beginning to allocate a a bunch of memory and this is about what isn't freed (how much it increases each time in Plesk).

 

How to free this memory used by ini_set?

 

And, by looking the code below, why am I ending with more RAM than the begining...and why 70mb at the begining at all?

 

Thanks so much!

 

<?php
set_time_limit( 0 );
echo ( memory_get_usage() / 1024 ) . ''; // 70mb
ini_set( 'memory_limit', '500M' );
echo ( memory_get_usage() / 1024 ) . ''; // 70mb
$bob = file( 'thebig60mbfile.txt' );
echo ( memory_get_usage() / 1024 ) . ''; // 350mb
unset( $bob );
echo ( memory_get_usage() / 1024 ) . ''; // 113mb
?>

 

[B0b]

I'll obviously donate to the forum if I get this solved 

I'm really lost, I tried again with a script that consumed over a gb of RAM, it did completly finished and erased all vars, but the RAM is still taken?

Any clue what could do this and how to fix it? I really need help :S

[oni-kun]

 

I'm really lost, I tried again with a script that consumed over a gb of RAM, it did completly finished and erased all vars, but the RAM is still taken?

Any clue what could do this and how to fix it? I really need help :S

 

http://www.webreference.com/programming/php_mem/

 

I'll obviously donate to the forum if I get this solved 

 

A bribe? Wow.

And i'm sure there's much more on Google.

[B0b]

I've read your whole link before posting this topic :S

It talks about memory in a lower level (Zend), instructive, but useless...

 

Yup, I'm bribing. I'm desperate: this is vital and urgent.

[oni-kun]

I've read your whole link before posting this topic :S

It talks about memory in a lower level (Zend), instructive, but useless...

 

Yup, I'm bribing. I'm desperate: this is vital and urgent.

 

So low level is no level to you?

 

70MBs is the processes itself, such as mysqld and apache etc. etc. From the manual:

 

    unset() does just what it's name says - unset a variable. It does not force immediate memory freeing. PHP's garbage collector will do it when it see fits - by intention as soon, as those CPU cycles aren't needed anyway, or as late as before the script would run out of memory, whatever occurs first.

 

    If you are doing $whatever = null; then you are rewriting variable's data. You might get memory freed / shrunk faster, but it may steal CPU cycles from the code that truly needs them sooner, resulting in a longer overall execution time.

 

http://php.net/manual/en/function.unset.php

 

Garbage collection is  something you should have read in the manual, you'll find your answers there.

[B0b]

Thanks for your help, but NULL doesn't seem to be better.

 

I've however read that doing this object oriented could be the solution (by destoying the class), what's your opinion?

 

Please reply, it's 00:40 here and I've been working on that script nearly non-stop for 48 hours (2-3 weeks I'm on it), I got strict delays and it would be fantastic to have a straight answer before I fall into coma 

I can't find much more about garbage collector in the documentation.

 

Thanks again.

[oni-kun]

Thanks for your help, but NULL doesn't seem to be better.

 

I've however read that doing this object oriented could be the solution (by destoying the class), what's your opinion?

 

Please reply, it's 00:40 here and I've been working on that script nearly non-stop for 48 hours (2-3 weeks I'm on it), I got strict delays and it would be fantastic to have a straight answer before I fall into coma 

I can't find much more about garbage collector in the documentation.

 

Thanks again.

 

Yes, an object can be destroyed or destructed much easier, but the problem is that you are expecting to remove the memory faster than it does.

 

You can try this here: http://php.net/manual/en/function.gc-collect-cycles.php , But other than unset()ing the variable, there is nothing else you can do for garbage collection.

[Buddski]

Have you noticed fluctuation in the RAM with any other scripts? How often will this memory sucking script run.. if its once in a blue moon and your finished RAM usage is only 113mb i would say that its not that bad..taking into account all the processes running on the server..

Have you taken measurements after a certain amount of time.. to let gc do its thing?

 

If its staying up around 113mb forever its not ideal but I dare say it will drop back down after a period of time..

[B0b]

gc_collect_cycles() isn't defined on the server?...

I am not expecting the memory to freed instantly, but it's been HOURS that it stay saturated! There must be something wrong.

 

How of an improvement are objects?

 

I just got the server and haven't used any other script: don't have any other of that nature anyways.

 

The problem isn't the RAM used shown by memory_get_usage(), but the RAM usage indicator in Plesk control panel (which you can see a screenshot of in my first post) which grow up of 1GB+ each time I run the script while never reducing. What's the meaning of that? It perhaps does nothing important in fact  :-\

 

Thanks again guys.

[oni-kun]

Is your PHP version of >= 5.3.0? If you're down at 4.3 there may be many more areas of the problem. Your plesk panel would be reporting the memory usage for the whole server's end, not what is available to PHP (memory_get_usage)

[B0b]

PHP 5.1.6 is installed: not too bad. There is solely the script installed/running on the server ($180+/month), so if I understand correctly your post, Plesk is showing up the RAM used by the script, which means I'm indeed screwed...But then, what is get_usage showing (memory used by PHP? - but there's only PHP running on there!). Headache is getting worse.

[Buddski]

No, Plesk will show the full server memory usage, NOT just the PHP scripts.

And php wouldnt be the only server process, youve got mysql, apache (or alternate), php and whatever else it needs.. Not to mention Plesk itself will be using memory..

[oni-kun]

Yep. How useful is the (~20?)MBs to you? I don't find it as a problem, again as long as it's not a process that is running every millisecond. Since you're over version 5.0 GC will do whatever, whenever it deems fit on to remove the extra memory caused by unset() or otherwise.

[B0b]

I mean, Plesk is showing I'm using 7GB+ and I got nothing else (beside default stuff) running than the script.

If an idle SQL, PHP and Apache takes that much memory to live, than I know several persons who must have problems.

 

Seriously, how could Plesk show several GB of RAM used and memory_get_usage() only 70MB in my circumstance?

 

PS: memory_get_usage() dramatically lower Plesk's RAM used 

 

My worry is that while having the server totally idling, RAM isn't emptied even after hours.

[Buddski]

If PHP's function is saying that its using 70mb and Plesk is saying 7gb something is wrong with Plesk or the server config itself... does plesk have a function that will break down its processes that make up that 7gb?

[oni-kun]

I mean, Plesk is showing I'm using 7GB+ and I got nothing else (beside default stuff) running than the script.

If an idle SQL, PHP and Apache takes that much memory to live, than I know several persons who must have problems.

 

Seriously, how could Plesk show several GB of RAM used and memory_get_usage() only 70MB in my circumstance?

 

PS: memory_get_usage() dramatically lower Plesk's RAM used 

 

My worry is that while having the server totally idling, RAM isn't emptied even after hours.

 

Maybe you want to set a CRON to log the memory usage when the server is idle for half a day and see what goes on there. But PHP should not be your problem, This may be your server's fault or another software taking up so much memory, PHP would not allocate that much for a 60MB file that is GC'd supposedly after.

 

I'd take it up with the techs..

 

EDIT: And yeah, if you can view processes (via shell, or panel) You SHOULD. This will be a very very very easy answer to your memory problem if you could.

[B0b]

I don't see how a CRON would help if PHP's function always tells me ~70mb right after I unset the vars?

I know! I know! I've Google'd intensively but can't find anything about Plesk's system processes, neither can find anything in the panel itself.

And the host (ServerLoft) doesn't seem to offer KVM by which I guess I could see server's processes directly from the OS.

 

So in short, technicians? I don't know.... doesn't sound like a server related issue? I'm really screwed  :-\

 

1:50 AM.... hawww..

 

EDIT: Budd, I got no clue!..

[oni-kun]

Well from what PHP is reporting, you know it is not PHP. So it's a tech support issue I suppose from now.

[B0b]

Well... to me it really looks like it's all about ini_set( 'memory_limit', '1G' ); (700M or whatever).

What Plesk is using is +- that value. It allocates all the memory I ask and never free it.

 

There must be something we don't understand correctly about ini_set. No?

 

If the above gives you an idea, please let me know.

Otherwise I'll check out tomorrow if what Plesk is showing is a problem at all, perhaps the script may run fine even when Plesk says it's full.

 

For now, I'm exhausted.

 

Thanks again guys for the great support.

 

[B0b]

I'm going to bump this as I'd love to hear more opinions.

 

In short, my server wont free the memory allocated by PHP's ini_set.

For example, if I use ini_set( 'memory_limit', '500M' ), 500MB is taken in Plesk control panel and is never freed.

 

Check out this test script:

 

<?php

echo ( 'Memory used at the beginning: ' . memory_get_usage() / 1024 ) . '<br/>'; // About 70mb is used.

set_time_limit( 0 ); // Removing script timeout.
ini_set( 'memory_limit', '500M' ); // Allocating enough memory for the script.
ini_set( 'display_errors', '1'); // Displaying any error if there is.
error_reporting( E_ALL );

for ( $i = 0; $i <= 2500000; $i++ ) // Filling an array of 2.5 million message.
{
$theArray[] = 'averylongmessagehere';
}

echo ( 'Memory used after filling the array: ' . memory_get_usage() / 1024 ) . '<br/>'; // About 350mb is used.

unset( $theArray );

echo ( 'Memory used after unsetting the array: ' . memory_get_usage() / 1024 ) . '<br/><br/>'; // About 90mb is used.

?>

 

After running this, 500mb is added to Plesk memory bar and is never removed.

 

Please, give me your opinions.