PHP array size limit?

[B0b]

Hi guys,

 

I am having an urgent and hard to solve problem here.

 

I can't seem to find why I'm having troubles with PHP and huge arrays (millions of 20+ char entries (actually 1 or 2 million)).

A client asked me to build up a script which needs, at first, to take his 60mb+ files and place them into an array.

He provided me a sick server only for this (8gb RAM, Xeon, etc, etc), so memory issue shouldn't be a problem.

 

I tried file() which just returns an empty array. I then tried a substitute manually reading each line of the file, same problem.

 

I then quickly wrote a little script going from 1 to 2 million and echoing 'finished' at the end: 'finished' never gets outputted.

And no error displayed!? It stops between 982000 and 982999.

 

What's wrong? Is there a parameter in php.ini or something limiting array size?

 

Thanks!

 

<?php

set_time_limit( 0 );

$testarray = array();

for ( $i = 0; $i <= 2000000; $i++ )
{
$testarray[] = 'testarrayelementof20ormorechar';
$tmpCount = count( $testarray );
if ( $tmpCount%1000 == 0 ) // Not to put too much load.
{
	echo $tmpCount . '<br/>';
	flush();
}
}

echo 'finished'; // This will never appear.

?>

[PFMaBiSmAd]

Just because the server has 8gb of memory, does not mean that much memory has been configured for php to use. Add the following two lines of code immediately after the first opening <?php tag to find out why the script is stopping -

 

ini_set("display_errors", "1");
error_reporting(E_ALL);

[Buddski]

Your PHP script is probably failing to finish due to a timeout error

turn on error reporting and you may get an error showing you this

error_reporting(E_ALL);
ini_set('display_errors',1);

If this is the case you need to increase the amount of time PHP allows the script to run

[B0b]

You guys are mind connected ;P

But it's PFMaBiSmAd who's right!

 

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 24 bytes) in /var/www/vhosts/heymman.com/httpdocs/phptest.php on line 13

 

Which is 128mb: quite a self explanatory number!

 

I assume this is in php.ini, any clue where more exactly?

 

Thanks again, that error reporting trick is fantastic, that was missing to my knowledge. Thanks!

[Buddski]

Its in your php.ini but if its probably best (my understanding) is to change the memory limit using ini_set inside the script itself.

 

ini_set('memory_limit','128M'); // or something larger

[B0b]

Oww...doesn't work!

 

From the PHP documentation, I see during installation the guy got to enable the option "--enable-memory-limit" and define functions memory_get_usage() and memory_get_peak_usage().

If I run phpinfo(), I can find "--enable-memory-limit" but not the two functions: I guess they aren't defined? If so, I got to reinstall PHP? How.....

 

Thanks!

[Buddski]

Are you getting the same error?

After your ini_set command use ini_get to see if that value has changed.

It might be possible that your script requires MORE than what you have specified.. we are talking millions of entries..

[B0b]

No, the error is the exact same with same amount of memory limit reached. Even if I put 700mb or 7gb... It just doesn't take the new limit in account.

PS: I know, 700mb = '700M' and 7gb = '7G'.

[Buddski]

does the ini_get function say that the value was changed?

[B0b]

Nope, still 128M!

 

ini_set( 'memory_limit', '700M' );
echo ini_get( 'memory_limit' ); // Echo '128M'

[Buddski]

Have a read of this.. it gives you other alternatives to changing your memory_limit..

You can either change it in the PHP.ini file or use .htaccess

http://www.prothemer.com/blog/hosting-and-security/how-to-change-your-php-memory_limit/

[B0b]

It works! Alleluia! iniset() works. The problem was that PHP on my domain set to "safe mode". Haa! You should remind that solution for further help

Just one more simple question if you would like to answer: if I want to remove limit at all, should I add -1 with quotation marks or not (-1 or "-1"), both seems to work!

 

Thank you so much for your help!

[Buddski]

I dont think it matters.. if they both work it clearly doesnt