Upload massive files with PHP (i.e. 1gb + ...)

[robcrozier]

Hi everyone, i've been informed that PHP is not ideally suited to uploading very large files (i.e. 1GB + etc...), and that something like perl would work better.  I was just wondering if anyone has managed to successfuly upload very large files with PHP?  If so how did you go about it and basically... would you recommend using PHP for this?

 

Cheers!

[Mark Baker]

It isn't particularly PHP or Perl that uploads files: it's the browser and web server using a POST request, which then makes them available to your CGI scripts. About the only control that scripts have over this is being able to reject the upload (once the http request has completed its part) based on criteria such as file size.

In that regard, your scripting language makes absolutely no difference whatsoever.

 

I'm working with uploaded files that are several GB in size, with PHP to process the file once the http upload request has completed.

My front-end script supports uploading several files at a time.... but I use Ajax to post each file one at a time so I'm not using all the limited connections at the client side (browsers like IE can only process 4 concurrent requests). All that the PHP upload script does is validate the upload and move the file from its temporary upload directory to a 'pending' directory, so that the http connection to the browser is freed up.

All significant processing of that file (loading data into the database), is handled by a cron task that executes a separate PHP script to handle that data.

I do also have an asyncronous feedback mechanism... as the file is actually processed by the back end cron task, its status is logged... so a user has a separate query that can tell them how the back-end processing for each file is progressing.