Out of memory

[sandy1028]

The code is reads the two files and sores into an array.

From the first file, 4th field and .xml is removed and is compared with the second file.

If the line exists in the second file, the line with 4th field and xml should be printed.

 

This works when the array size is small.

Suppose there are 230K above elements in the two files, it gives out of memory error.

 

 

 

How can I optimise the code, read the two files, and printin the lines, if it exists in first file

 

 

file1 contains

 

2000/03/01/aaa/aaa.xml
2003/05/11/ccc/bbb.xml
2008/08/05/bbb/ccc.xml

 

 

file 2 contains

2000/03/01/aaa

 

<?php

$input_path = $argv[1];

        $content_inputfile = file($input_path);
        foreach ($content_inputfile as $content_input)
        {
                $content_input=rtrim($content_input);
                $data= explode("/",$content_input);
                $data[4] = preg_replace('/\.xml/','',$data[4]);
                $finalarr_delimited[] ="$content_input#$data[0]/$data[1]/$data[2]/$data[4]";
                $finalarr[] ="$data[0]/$data[1]/$data[2]/$data[4]";
        }

$dbfile_path = $argv[2];
        $content_dbfile = file($dbfile_path);
foreach ($content_dbfile as $content_db)
{
$str=strlen($content_db);
$content_db=rtrim($content_db);
$content_dbfil[]=$content_db;
sort($content_dbfil);
}

$differnece = array_diff($finalarr,$content_dbfil);

foreach ($finalarr_delimited as $line){
$position = strpos($line,"#")+1;
$lines = substr($line,$position);
if(in_array($lines,$differnece)){
$diff[]=$line;
}
}

echo "Checking the string in diff array\n";
foreach ($diff as $final){
$lines = explode("#",$final);
print "$lines[0]\n";
}



?>

[celsoendo]

<?php
$myFile = file("file.txt");
?>

 

You can't do this never for one simple reason: memory.

The function file reads the entire file an put it on an array. Arrays are stored on memory, memory is not unlimited, so memory blows up!

 

The best to do is reading the file with a handle.... something like this:

 

<?php
$myFile = "file.txt";

// Open the file for read only from starting
$myHandle = fopen($myFile, "r");

while (!feof($myHandle)) {
    // Read the line
    $myLine = fgets($myHandle, 8024);
    print $myLine ."\n";
}

fclose($myHandle);
?>

 

With this you are reading line by line... not the entire file all at once. The only limitation for this technique is the max_execution_time variable on your php.ini. Default is 30 seconds. If you need, you can put set_time_limit(0) on the top of your code and then no time limit will be imposed, then you can read any file of any size.