[SOLVED] Storing large amounts of files

[pocobueno1388]

This question is coming from pure curiosity. I'm not looking to create a site that stores large amounts of files from users or anyting, but I am curious how it is done.

 

I am wondering how sites such as youtube can store such huge amounts of video files and where they store them? I am assuming they store the filename in the database that connects it to which user posted it, when it was posted, etc...and then they store all the files in a directory.

 

How can they store millions and millions of files without running out of space? Also, do you think they make a new directory for each user, or are all the files just all jumbled into one folder?

 

If anyone can shed some light on this for me, that would be great =] It just boggles my mind on trying to figure out how this is done.

 

Thanks

 

[taith]

um... the only way of never running out of space, is never running out of money to buy more space ;-) one of the perks of being last years most popular page ;-)

[pocobueno1388]

Haha, yeah...kinda figured that.

 

I am still a bit curious about the structure in how they organize all these files though. Anyone care to give their thoughts on this?

[yzerman]

http://en.wikipedia.org/wiki/Data_center

 

Basically, its a room full of computers - and lots of storage space.

 

If you are lucky, you have a data center near you that will give you a tour and explain exactly how they do what it is they do. When I went to dallas last year, I was given a tour of one of ThePlanet's Data Centers. I was in utter awe at the shear amount of information that was running thru that data center. I know I really didn't answer your question, but the truth is, I don't know the answer.

[rcorlew]

Personally, I use private directories for each user. Then I create sub-directories and so on.

 

Then one benefit of that is if you keep track of logins and pageviews and stuff for each user/page, you can do some house cleaning every month by generating reports as to which file has not been seen in 6 months, which user has not logged in for over a year.

 

Once you have that info, you can simply delete said user/directory and bingo, more space.

 

Space costs money, the delete key is free, hahaha.

[seb hughes]

Youtube probs has a couple hundered servers, and they probely use load balancing to balance out there traffic to keep the site going, I doubt each user actually has a folder.

[pocobueno1388]

Thanks everyone, that basically answers my question =]