Jump to content


Photo

Large File Uploads/Downloads


  • Please log in to reply
7 replies to this topic

#1 koencalliauw

koencalliauw
  • Members
  • PipPipPip
  • Advanced Member
  • 36 posts

Posted 26 April 2006 - 09:17 PM

Hi,

I'm working on a new project where I need to be able to upload and download large files. The server has loads of bandwidth, but only limited RAM (512Mb), so I must avoid files being loaded into RAM while uploading or downloading. I have a simple PHP/MySQL setup and I do have root access to the server, so setting ini vars is no problem.

Large files = upto 200Mb

Questions:
How do I upload large files without filling RAM?
How do I download large files without filling RAM?
Is it a good/horribly bad idea to try and store the files in the DB?
Is it a good/horribly bad idea to base64 encode the files for future compatibitly (XML,...)?
In the solutions to above problems, is a progress bar possible?

Side Question:
Is there a decent way to detect mime types for the uploaded files or do I have to rely on the file extension?

I know these are a lot of questions all at once, but these problems have been bugging me for a while now, and no other forums give a decent solution, so I have to rely on you pro's to give me some answers.

Thank you for replying.
Best regards,
Koen Calliauw

#2 DepretioN

DepretioN
  • Members
  • PipPip
  • Member
  • 12 posts

Posted 26 April 2006 - 09:22 PM

Linux or Windows?

And do you just need to transfer these files to and from servers? Or are you pushing the files to users?

#3 koencalliauw

koencalliauw
  • Members
  • PipPipPip
  • Advanced Member
  • 36 posts

Posted 26 April 2006 - 09:25 PM

[!--quoteo(post=369007:date=Apr 26 2006, 11:22 PM:name=DepretioN)--][div class=\'quotetop\']QUOTE(DepretioN @ Apr 26 2006, 11:22 PM) View Post[/div][div class=\'quotemain\'][!--quotec--]
Linux or Windows?

And do you just need to transfer these files to and from servers? Or are you pushing the files to users?
[/quote]
Linux Fedora with Customised Plesk

Uploads: users to server
Downl: server to users

#4 DepretioN

DepretioN
  • Members
  • PipPip
  • Member
  • 12 posts

Posted 26 April 2006 - 09:33 PM

For pushing files to your users, rather put the files in a directory like bin , or files ... and then in your code just use header("Location: files/filename.zip"); to start a normal download for the user rather than using php to push it.

For uploads, I'm not sure if PHP loads it into memory or not, but you will need to change the upload_max_filesize directive in your php.ini to cater for 200MB.

Edit: For large files you are always better off using ftp.

#5 koencalliauw

koencalliauw
  • Members
  • PipPipPip
  • Advanced Member
  • 36 posts

Posted 26 April 2006 - 09:41 PM

[!--quoteo(post=369012:date=Apr 26 2006, 11:33 PM:name=DepretioN)--][div class=\'quotetop\']QUOTE(DepretioN @ Apr 26 2006, 11:33 PM) View Post[/div][div class=\'quotemain\'][!--quotec--]
For pushing files to your users, rather put the files in a directory like bin , or files ... and then in your code just use header("Location: files/filename.zip"); to start a normal download for the user rather than using php to push it.

For uploads, I'm not sure if PHP loads it into memory or not, but you will need to change the upload_max_filesize directive in your php.ini to cater for 200MB.

Edit: For large files you are always better off using ftp.
[/quote]

Hi,
thanks for your reply, I should have mentioned that this particular website is extremely permission driven, could I still use the download method you described but first have my user authenticate? If I redirect my user to the file itsself, could I then secure the files folder so that it cannot be accessed directly (so only using the script)?

The website in itsself will be used by total newbies, so I want to stay away from FTP and Java applets for the uploading, but maybe there is a way to use the ftp functions in php to put the file on the server? If so, the RAM question arises again... :(

Best regards,
Koen


#6 koencalliauw

koencalliauw
  • Members
  • PipPipPip
  • Advanced Member
  • 36 posts

Posted 27 April 2006 - 06:53 AM

anybody have any ideas on how to upload large files decently using (preferably) only php?

#7 ober

ober
  • Staff Alumni
  • Advanced Member
  • 5,337 posts
  • LocationEast Coast, USA

Posted 27 April 2006 - 12:22 PM

I'm not sure why you think these files are loaded into RAM when you're uploading them. I'm not 100% sure, but I don't think that's the case.

To answer some of the other questions, I don't think it's ever a good idea to store images in a database. It's a DATAbase... not a FILEbase. Data meaning text and small binary strings, not large chunks of images and random files.

I personally wouldn't bother encoding the files. Let the user deal with that.

Progress bars are very possible. Consider using something like AJAX to accomplish that.

And I would definately consider using the PHP FTP functions instead of the normal upload functions.

Info: PHP Manual


#8 koencalliauw

koencalliauw
  • Members
  • PipPipPip
  • Advanced Member
  • 36 posts

Posted 27 April 2006 - 03:15 PM

Hi,

thanks for replying, I'll post the upload script here when it's done.

Koen.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users