Mass Data via PHP

[e33basketball]

Hi,

 

I have a large amount of data, around 9MB, in a csv file. I have a script to convert to SQL and insert into a database. However, I cannot ever get it to finish, it  sits there for over 30 minutes, with no output. I've tried upping the allowed RAM usage in  the php.ini file as well as the time to execute and other parameters along those lines.

 

Does anyone know a good solution this? Would something like this be possible via command line?

 

Any suggestions would help,

 

Thanks!

 

Erik

[aniesh82]

hello,

 

To do this set the execution limit to infinity. This can be achieved by the following line:

 

set_time_limit(0);

 

I think you already done it.

 

It takes less than half an hour.  So please check your script with a csv file of small size to know whether it is working perfectly or not?

 

Did you output anything after the completion of your insertion?

 

If any queries regarding this, please reply. I am handling csv files with more than 23 MB size. So do not worry.

 

If you want, please go through the following lines of code.

 

<?php

set_time_limit(0);

error_reporting(E_ERROR ^ E_WARNING);

include("connection.php");

 

 

$city = "12.csv"; /* files to read */

$handle = fopen($city,"r");

 

 

while(($data = fgetcsv($handle, 1000, ",")) !== FALSE)

{

$countryCode = $data[0]; /* specify the csv fields one by one */

$cityName = $data[2];

$latitude = $data[3];

$longitude = $data[4];

 

$cname = $arr[$countryCode];

 

$sql = 'INSERT INTO nameCityNew (countryCode ,countryName, cityName , latitude, longitude  ) VALUES ("'.$countryCode.'", "'.$cname.'", "'.$cityName.'","'.$latitude.'","'.$longitude.'")';

 

mysql_query($sql);

 

}

echo "<br> Completed Execution ";

 

?>

 

Regards

Aniesh Joseph

[email protected]

[npsari]

Try changing you explorer

 

If you have Firefox... use it

 

that might make it work this time

[smc]

Try changing you explorer

 

If you have Firefox... use it

 

that might make it work this time

 

That wouldn't do anything... PHP is server side, not client side.

[npsari]

cus i was trying to backup my databse, and it stucks all the time

 

when i changed the explorer, it worked

[smc]

Well you must have done something on the server or the server did something because PHP is entirley server side, I could unplug my monitor and throw it out the window and it wouldn't affect what the script parsed.

[Psycho]

Well you must have done something on the server or the server did something because PHP is entirley server side, I could unplug my monitor and throw it out the window and it wouldn't affect what the script parsed.

 

But, if you didn't have a monitor to view the results, how would you know? LOL.