Script is timing out on server

[n1concepts]

Can someone propose a solution to resolve the following:

 

I have a PHP script that runs just fine when processing small amounts of data (opening and reading CSV file and performing afew validation checks before saving to db).

 

Issue: when attempting to process a large file, I have found the execution time exceeds the "timeout" value currently set on the server which is 60 seconds.

Note: I do not want to increase this time interval as that's a security risk so need to figure out a way to break up the data - being read from the CSV file - into chunks and some way create mini-processes (I guess) to execute chunks of that data at a time until the entire file is read.

 

Any suggestions?

[Adam]

If it's a time-out issue then you're not going to get around it by breaking it down -- in-fact although this can save memory, it generally increases the time needed. You can increase the time-out limit for that request only, which isn't really a security issue if you're expecting it to take longer, by using set_time_limit.

 

 

[n1concepts]

Yeah, I modified the code and you are exactly right as that's what I'm seeing.

Ok, there is something I can do to reduce the time:

 

1. instead of processing the records (one-at-a-time) validating before logging into main table, I can simply import all into a temp table, then run through that temp table (altering) any that needs validation accordingly with a different script/process.

 

That will reduce the overall time of any one script and load on resources.

 

Appreciate your quick answer - PEACE!