memory usage & php

[surion]

hi,

 

today i have written a script (using the safox api) to convert data from a mysql database to several xml files.

we re talking about quiet alot of data.

When i run the script it goes incredibly fast, it creates like 650 xml files in 20 seconds (while with my earlier script not using sofax it took like 20 minutes for the same amount of data).

anywayz, i got a new problem now

 

after it ran 1 minute and processed about 1200 xml files, i get the next error:

 

Fatal error: Allowed memory size of 524288000 bytes exhausted (tried to allocate 52 bytes) in C:\www\xmlswitch\classes\safox_g.cls.php on line 767

 

i know what it means, and i know that i could solve the problem with messing around with ini_set or just change the max memory size in php.ini, but that is what to avoid doing.

 

do you guys got any tips on saving memory usage while using php?

is there something like .net's "garbage collection" in php?

or is there any other way to save memory ?

[Wuhtzu]

Just get rid of stuff you do not need any more. Unset variables when they are no longer in use ect.

 

http://php.net/manual/en/function.memory-get-usage.php

[surion]

i just changed the script a bit

everywhere unsets for vars i don't need anymore

 

yet what s very odd is that if i output the memory usage it only raises :s never goes down :s:s

 

i ve also put error reporting on E_ALL now to see if there were any notices orso that got written to the log spending memory, but there weren't any,...

[surion]

i ve uploaded the script to http://www.lodedesplancke.be/temp/index2.txt (too big to post here i think)

can anyone take a look what could still be the big memory-spender?

 

oh and another thing:

script always runs out on the memory limit of .5 GB, but there s only a database of 20Mb and all the xml files together should only be around 15Mb :s

[btherl]

A few things

 

- You don't unset mysql query results, you call mysql_free_result() on them.  You can unset them as well but that won't really affect memory usage.

- Your function is quite long   A simple way to save memory is to put more work into functions.  A nice side-effect of this is that function local variables are freed automatically when the function finishes.

- Are you using memory_get_usage() or the windows alternative for measuring?  It makes a difference, as memory_get_usage() only tells you memory allocated by php itself and not by C extensions.  The alternative method tells you ALL allocated memory.

[surion]

A few things

 

- You don't unset mysql query results, you call mysql_free_result() on them.  You can unset them as well but that won't really affect memory usage.

- Your function is quite long   A simple way to save memory is to put more work into functions.  A nice side-effect of this is that function local variables are freed automatically when the function finishes.

- Are you using memory_get_usage() or the windows alternative for measuring?  It makes a difference, as memory_get_usage() only tells you memory allocated by php itself and not by C extensions.  The alternative method tells you ALL allocated memory.

 

-changed the unset of the results to mysql_free_results, as you said, didn't affect the usage

-i know, but i can't pull the xml things appart i think, if i do so i d have to use either references or globals, both spending more memory than i spend now

-as you can see in the function i made sure that both could work, now i m testing on a windows machine, so it ll be the windows alternative

 

i realy don't understand where that .5Gig memory usage comes from :s:s:s data is not that big at all :s:s and i also still got no clue why unsets or free_results does not make the memory usage drop at all :s:s:s

[surion]

after alot more testing on this issue it turns out that my script isn't the problem, but the safox api is the big memory user, any safox-wizards in teh house? the api can be found @ http://sourceforge.net/projects/safox/