Huge arrays eating up ram on Linux, but not Windows?

[soadlink]

Hello,

I am working with the following script that loads a list into an array and then echos each word in the array on a new line:

 

<?php

$loadlist = array_map("rtrim", file("c:\\list.txt"));

foreach ($loadlist as $word) {
echo ($word . "\n");
}

?>

 

I run my scripts with the CLI (not on a website), and it works great in Windows XP when loading lists as large as 375,000 words, and I can run plenty of instances of it without lag issues (8+ command lines going at once). But when I try to run the same script on Ubuntu I get the following error: Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 13491373 bytes). But that can easily be fixed by setting a larger memory limit such as: ini_set("memory_limit","75M");

 

However I only have to up the memory limit on Ubuntu, but not Windows. And when I do up the memory limit on Ubuntu, only 1 or 2 instances can be run before the machine starts to bog down and get really slow. I'd assume this is because php is trying to load the entire list into memory in Ubuntu, but I am curious as to why Windows runs it fine.

 

The PCs are the same (I just removed the Windows install recently and put Ubuntu on there), and it has 1GB of ram, and an AMD64 3200+ processor. I'm just wondering if there is something I need to change to get the script to run like it did Windows, without being a memory hog or having to increase the memory limit (since I didnt have to on Windows). Thanks for the help 

[effigy]

I cannot answer the Windows vs. Linux issue, but from the looks of it, that's a bad approach (unless PHP does something behind the scenes that I'm not aware of) because you're filling up an entire array only to output the data, i.e., you don't need an array. What about opening the file and going line by line to process and output the data? Something like this.

 

What does this give you? cat /proc/meminfo

[soadlink]

Yea it is doing stuff with the words on each line, I just had it stripped down (just echoing) for easier understanding. I guess I don't need an array, and could look at alternative methods for loading large lists and processing them. I was just hoping there was an option I could change so it doesn't load the entire list into memory as it appears to be doing, like Windows appears to do correctly.

 

I'll try the solution in that link you gave and see how it works, thanks! 

 

What does this give you? cat /proc/meminfo

 

MemTotal:      1035672 kB

MemFree:        118592 kB

Buffers:        177008 kB

Cached:        576840 kB

SwapCached:          0 kB

Active:        626848 kB

Inactive:      240496 kB

HighTotal:      131008 kB

HighFree:          592 kB

LowTotal:      904664 kB

LowFree:        118000 kB

SwapTotal:    3028212 kB

SwapFree:      3028212 kB

Dirty:            180 kB

Writeback:          0 kB

Mapped:        154300 kB

Slab:            39084 kB

CommitLimit:  3546048 kB

Committed_AS:  467732 kB

PageTables:      1748 kB

VmallocTotal:  114680 kB

VmallocUsed:      5724 kB

VmallocChunk:  108700 kB

[btherl]

I'm very surprised it doesn't use huge amounts of memory in windows, since you are instructing php to read the entire file in.  There's no way that can be optimized away.

 

The script says

 

1.  Read in the entire file

2.  Apply rtrim to every line

3.  Iterate over the array

 

Even the smartest interpreter can't avoid allocating that memory.. it's very strange indeed.