Whats wrong with having a lot of files on web server?

[NaurisSkulme]

I'm developing a web page that will allow users to upload files (word documents, presentations) on server and then access it. So far the system is this - save file in one directory and save path in database and when retrieving the file you get path from db and download it.

 

I havent encountered this problem personally but i am aware that storing a large number of documents in same place may cause major slowdowns.

 

There is a lot of information on internet that this problem exists. But im having a hard time finding exactly:

 

- what causes the problem ?

(is it too many files in one directory, is it too long file names??)

- where is the source of this couse ?

(is it OS, is it PHP - im using Zend Framework which uses PHP to develop this page)

- when does this problem becomes relevant?

(how many files do i have to have on system to withness this problem?)

- what are the solutions?

(renaming files, making subdirectories?)

 

Is there someone who knows theses things. If there is please answer 4 questions mentioned above (or just give me a link if its already been answered)

 

 

Thanks in advance

 

P.S. On another forum this one guy suggested that as long as i store file paths in database i can store millions of files in one directory it doesnt matter. is that true?

[Silverado_NL]

Hi im no filesystem expert or anything but hope this answers your questions.

 

Whats wrong with having a lot of files on web server?

nothing really, its what u do to them that matters.

 

a project that i have done some maintenance on in the past, saved file images to 1 location.

this directory had about 25.000 files in it and everything was still running smoothly.

 

what causes the problem ?

 

usually when a script try's to read the entire directory at once it slows down the operating system(the larger the directory, the longer it will take)

i have even seen some scripts that did a foreach loop for every file in a directory, which will be killing for ANY webserver.

 

- where is the source of this couse ?

 

bad coding or bad file/directory handling.

 

- what are the solutions?

renaming files surely will not help.

As for making subdirectories will only help in situations where ur reading the content of a directory.

making subdirectories will not make ur system read files faster.

 

P.S. On another forum this one guy suggested that as long as i store file paths in database i can store millions of files in one directory it doesnt matter. is that true?

 

pretty much yeah.

what he probely means is instead of reading the entire directory each time for the filenames u need, u can also store these filenames in the database. and call them when needed.

accessing a database is ALOT faster then reading from disk.(saves alot of system resources,even on small scale projects.)

 

although i dont support his idea of including the filepath in the database.

as this will make u need to change every path of every file if u ever decide to move your big directory.

 

hope this helps