[SOLVED] Working with large lists

[soadlink]

Hello,

I am working with processing large lists of words (over 10mb for some!), and they have 1 word or string on each line. I WAS using a different method, but it was eating up my memory because it would load the entire list into memory, so I was forced to take a different approach. I'm using command line for this. Here is my attempt:

 

<?php
$file = 'largelist.txt';
$handle = @fopen($file, "r");
if ($handle) 
{
	while (!feof($handle))
		{
			$buffer = fgets($handle);
			echo md5($buffer) . "\n";
		}
fclose($handle);
}
?>

 

So the code should echo the md5 hash for each list item on a new line, but the problem is that the MD5s are not the correct when it does. Only the very last md5 processed in the list is correct. I believe this may have to do with hidden line breaks being inserted.  I would like to be able to decide when line breaks are used.

 

So basically what I am looking for is: to be able to process each word in the list as I wish, but only the word! Not anything else like line breaks or hidden chars; I choose when the line breaks \n come in if I need them! Also, it would be nice to use the same variable each time around in the loop for the current word in the list (i.e. in my example I was able to use $buffer for each list item). This is probably a given though.

 

Thanks, and if I confused anyone let me know, and I'll try to explain it better

[soadlink]

I really need to search harder: rtrim()

 

The rest works as I want it to now