[SOLVED] explode() not working?

[unkwntech]

I am trying to parse a tab delemited flat file so that it can be imported into a MySQL database.

$fp = fopen("mls/Data/Listing04112007.log", "r");
while ($record = fget csv($fp, 5000)){
$new = explode("\t", $record);
//The following lines are so that I can see that it is working correctly
echo $new[0];
echo $new[1];
}

unfortunatly all it returns is

ArrayArrayArrayArrayArrayArrayArrayArrayArrayArrayArrayArrayArrayArrayArrayArray...

The page can be viewed here http://www.unkwndesign.com/tests/index.php

[veridicus]

If you're using fgetcsv you don't need to explode on the line.  It returns one line parsed into an array.

[Glyde]

If you want to use fgetcsv on a tab delimited file, use: fgetcsv($fp, 5000, "\t").  If you use this in place of simple fgetcsv($fp, 5000) you won't need an explode statement.  By the way, the reason this is happening is because fgetcsv creates an array of all of the CSV elements, where CSV = comma separated value, hence why you need to specify "\t" as the delimiter in the function, rather than the default comma.

[unkwntech]

Thank you for your replies.  However it still seems to have an error, the new code looks like this:

$fp = fopen("mls/Data/Listing04112007.log", "r");
while ($record = fget csv($fp, 20, "\t")){
echo $record[0];
}

[Glyde]

Any reason why you put a space between fget and csv?  It should be fgetcsv not fget csv.

<?php
$fp = fopen("mls/Data/Listing04112007.log", "r");
while ($record = fgetcsv($fp, 1000, "\t")){
     print_r($record);
}
?>

[keeB]

try this.

 

<?php
$fp = fopen("mls/Data/Listing04112007.log", "r");
while ($record = fget csv($fp, 20, "\t")){
print_r($record[0]);
?>
}

That will give you the array structure.

[unkwntech]

That was a typo. it does read fgetcsv. and as you can see here http://www.unkwndesign.com/tests/index.php it is parsing the file and it is removing the tabs. but not putting it into an array.

[Glyde]

Well, replace the print $record[0] with print_r($record) as I stated above, and see what happens.

[unkwntech]

Now it outputs the array. Example: Array ( [0] => LN [1] => HSN [2] => STR [3] => MUN [4] => COU [5] => S ) Array ( [0] => TATE [1] => ZIP [2] => LP [3] => MAR [4] => MPY [5] => ) Array ( [0] => MPX [1] => SDN [2] => SDP [3] => DVN [4] => UNI [5] => ) Array ( [0] => CW [1] => STA [2] => OWN [3] => ST [4] => TYP [5] => LA ) Array ( [0] => G [1] => LO [2] => SBA [3] => FBT [4] => HBT [5] => QBT ) Array ( [0] => [1] => TBT [2] => TBA [3] => BR [4] => BL1 [5] => BD1L ) Array ( [0] => [1] => BD1W [2] => BL2 [3] => BD2L [4] => BD2W [5] => ) Array ( [0] => BL3 [1] => BD3L [2] => BD3W [3] => BL4 [4] => BD ) Array ( [0]

it is still availible at the link i posted

[Glyde]

Well you have the maximum line length set to 20 characters...try this:

<?php
$fp = fopen("mls/Data/Listing04112007.log", "r");
while ($record = fgetcsv($fp, 0, "\t")){
     print_r($record);
}
?>

[unkwntech]

It is producing the same results.

[unkwntech]

How could I read this file line by line but still break the line into an array so that I can import each line as a single record into MySQL?

[Glyde]

How about:

<?php
$lines = file("mla/Data/Listing04112007.log");
foreach ($lines as $line) {
     $lineData = explode("\t", $line);
     print_r($lineData);
}
?>

My assumption is that that log file isn't a valid CSV (technically TSV) delimited file, and therefore fgetcsv is unable to parse it correctly.  I personally have never used fgetcsv on anything other than a comma-delimited file, and even with that it was only once.

[unkwntech]

Now all it outputs is a blank page.

[unkwntech]

IDK whats wrong it appears that the page is not being parsed by php.  (I am being asked to download it.) but other php files on that server are working just fine.

[unkwntech]

 

Fatal error: Allowed memory size of 83886080 bytes exhausted (tried to allocate 1136 bytes) in /home/unkwn/public_html/tests/index.php on line 7

 

 

This error does not even make sense tried to allocate 1136 bytes but it used more then 83886080?

[AndyB]

How big is this file? Any chance you can post a sub-set of it here (as a text file attachment) so anyone interested can experiment with it?

[unkwntech]

After changing the Maximum memory limit of PHP dangerously high it now outputs the following:

Array ( [0] => Array ) Array ( [0] => Array ) Array ( [0] => Array ) Array ( [0] => Array ) Array ( [0] => Array ) Array ( [0] => Array ) Array ( [0] => Array ) Array ( [0] => Array ) Array ( [0] => Array )

[unkwntech]

How big is this file? Any chance you can post a sub-set of it here (as a text file attachment) so anyone interested can experiment with it?

The files average 60MB

http://www.unkwndesign.com/tests/mls/Data/Listing04112007.log

this is a direct link to the file.

[AndyB]

I don't think anyone will want to start a test by downloading a 60 Mb file 

 

How about a sub-set of the file not larger than 100 kb as a test download?

[unkwntech]

I have included only the first 5 lines.

 

[attachment deleted by admin]

[AndyB]

<?php
$stuff = file("Listing04112007.txt"); // read into array
for ($i=0;$i<count($stuff);$i++) {
    echo $stuff[$i]. "<br/><br/>";
$more_stuff = explode("\t",$stuff[$i]);
// only interested in first few array elements ...
        for ($j=0;$j<6;$j++) {
	    echo $j. " ->". $more_stuff[$j]. "<br/>";
	}
echo "<hr/>";	
}
?>

 

That works fine for me.  Maybe you can build from that ...

[Glyde]

I think memory issues are a huge problem here.  60MB for a text file that you're actually trying to iterate through isn't exactly easy.  62914560 bytes are trying to be parsed by PHP.  While PHP can handle this, it all depends on the quality of the server as well as PHP's memory settings and the amount of available RAM.  Are you sure all of the content in the file is correctly delimited by a tab?

[AndyB]

http://www.halfadot.com/temp/short-version.zip

 

I 'cheated'.  Imported the 60 Mb file into Excel as a tab-delimited file, removed all of the fields beyond the sixth column, and saved it back as a tab-delimited file (which is zipped as the link above) with 61,000 records or so.

 

It's still a big file when unzipped - 3+ Mb - but that might get you going forward.

[unkwntech]

Just curious as to why you removed everything after the 6th column?  And does anyone have any sugestions as to how I can acomplish this with the entire file?