affordit Posted March 19, 2011 Share Posted March 19, 2011 I am trying to split large csv file into manageable files to upload to database, I wrote this code and it does the job but I need to speed it up to get around the 30 second time out in php can anyone help? <?php $files = glob('*.txt'); foreach ($files as $file) { $handle1 = fopen($file, "r"); $size = filesize($file); if($size > 20000000){ $line=1; $handle2 = fopen($line.$file, "w"); while (($data = fgetcsv($handle1, 10000, ",")) !== FALSE) { $row = ($data[0]."[,]".$data[1]."[,]".$data[2]."[,]".$data[3]."[,]".$data[4]."[,]".$data[5]."[,]".$data[6]."[,]".$data[7]."[,]".$data[8]."[,]".$data[9]."[,]".$data[10]."[,]".$data[11]."[,]".$data[12]."[,]".$data[13]."[,]".$data[14]."[,]".$data[15]."[,]".$data[16]."[,]".$data[17]."[,]".$data[18]."[,]".$data[19]."[,]".$data[20]."[,]".$data[21]."[,]".$data[22]."[,]".$data[23]."[,]".$data[24]."[,]".$data[25]."[,]".$data[26]."[,]".$data[27]."[,]".$data[28]."[,]".$data[29]."[,]".$data[30]."[,]".$data[31]."[,]".$data[32]."[,]".$data[33]."[,]".$data[34]."[,]".$data[35]."[,]".$data[36]."[,]".$data[37]."[,]".$data[38]."[,]".$data[39]."[,]".$data[40]); if(strlen($row)>10) { $row = str_replace('"','',$row); $row = explode("[,]", $row); fputcsv($handle2, $row); $line++; if($line == 100000){ $handle2 = fopen($line.$file, "w"); } if ($line == 200000){ $handle2 = fopen($line.$file, "w"); } if ($line == 300000){ $handle2 = fopen($line.$file, "w"); } if ($line == 400000){ $handle2 = fopen($line.$file, "w"); } if ($line == 500000){ $handle2 = fopen($line.$file, "w"); } if ($line == 600000){ $handle2 = fopen($line.$file, "w"); } if ($line == 700000){ $handle2 = fopen($line.$file, "w"); } if($line == 800000){ $handle2 = fopen($line.$file, "w"); } if ($line == 900000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1000000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1100000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1200000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1300000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1400000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1500000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1600000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1700000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1800000){ $handle2 = fopen($line.$file, "w"); } if ($line == 1900000){ $handle2 = fopen($line.$file, "w"); } if ($line == 2000000){ $handle2 = fopen($line.$file, "w"); } if ($line == 2100000){ $handle2 = fopen($line.$file, "w"); } } } } } fclose($handle1); fclose($handle2); ?> Thanks everyone Quote Link to comment Share on other sites More sharing options...
creata.physics Posted March 19, 2011 Share Posted March 19, 2011 I wouldn't know anything about cvs files specifically or how to upload them into a db, but I must ask, have you taken out all of the code within the while loop to see if the loop itself is or isn't what's making the page timeout? that's just some basic debugging to see where exactly it's timing out at. If it actually is the time of porting that information over, you'll need to edit your php.ini file to change the script timeout time. Quote Link to comment Share on other sites More sharing options...
affordit Posted March 19, 2011 Author Share Posted March 19, 2011 I have edited the php.ini on my server but I am uploading these datafeeds to another server that I cannot change the .ini file on. The script does not time out on mine and I have already written a script to upload the files to the db but if the file is larger than 20,000,000 bytes it times out. I would just like to know if there is a cleaner way to write the code I have. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.