Avoid multi directory structure for userbase

[Zane]

So, right now, I'm working with a project that can potentially have thousands of users logging into it.  At this time, it is simply in development, maybe 3 users at the moment.  Regardless, the goal is to give users access to an online file editor, and in this editor, they can only see and edit their own files and folders.  No one else's.

The issue at hand, is I'm trying to (I want to) avoid having to create a subdirectory for every user registered.  However, I'm having a hard time wrapping my head around an idea and infrastructure for organizing something like this.  My first thought was to use a virtual host, but that would still entail have a directory for every single user, and having that many subfolders to maintain would be a nightmare.  What if the configuration files' structure has to be changed?  Then, a script would need to run to loop through each and every user's subdirectory to update that configuration file.  Of course, I suppose there could be a single config file that all user's are tied to.

I realize there isn't a one shoe fits all for this design, but I know this isn't the first project in the world to have this problem.  I'm making every effort possible to keep scalability in mind.

**These will not be Linux users or even Windows users, they will be entries in a users table in a MySQL database; just your run-of-the-mill CMS.  Surely Wordpress/Github/Blogpost/etc don't create a new subfolder for every user registered?

Ideally, at the surface, it would appear that each user has their own directory, but behind the scenes there would be a system in place that is only rewriting the URL to appear that way.

www.website.com/user1
www.website.com/user2
and so on.

 

When in reality, the actual URLs before being rewritten might be

www.website.com?uid=123
www.website.com?uid=4833
and so on...

 

Although I'm aware of how to setup something like that, I'm just not convinced is the most efficient solution.  As it is, users will have their how row in the users table in the MySQL database, and there would most likely a be a table for files and folders with a foreign key for the user_id found in the user table.  (I haven't taken into account the possibility of groups yet, that would be down the road). 

Anyway, I'm just curious as to what kind of solutions people may or may have not  designed into their current systems that address the ability to avoid thousands of subfolders for thousands of users..   Please let me know if I need to provide more details.  As I have posted this in the Application Design forum, this is mostly a theory based question.  I'm not expecting a concrete copy and paste solution.  I just need to warp my head around a potential solution and how I could make it work.

[kicken]

It sounds like what you're describing / debating is a how to create a multi-tenant application, is that right?  Essentially you have an application that is dynamically configured and has separate data for each user instance (ie, several different wordpress blogs sharing the same code base).  I'm not sure I see the need for such complexity based on what is mentioned.

2 hours ago, Zane said:

the goal is to give users access to an online file editor, and in this editor, they can only see and edit their own files and folders.

That just sounds like basic user management / access to me.  You have your users table and a files table and associate the files with their users.  How you physically store the files on disk isn't really that relevant. Just ensure you code your editor so it can only load files the user has access to.

2 hours ago, Zane said:

Ideally, at the surface, it would appear that each user has their own directory, but behind the scenes there would be a system in place that is only rewriting the URL to appear that way.

Is there some public aspect to this where you need to isolate the user's files based on just the URL and not the user being logged in?  Last time I did something like this, it was based on the host name, but a URL directory would work as well.  Using the host name could have allowed the end-users to apply their own domain name to application instead of having to use mine (though that was never actually done).

 

Based on the info provided so far, I'd probably just be doing essentially what I think you're already considering, just without the creation of separate folders for users bit.  I'd just store all the files for all users in one directory structure somewhere with random names.  The database would contain the original file name/mime type and user association information.  I'd use either mod_rewrite or FallbackResource in apache to handle public access to the files and normal login procedure for the editing/uploading bits.

[Zane]

Quote

I'd just store all the files for all users in one directory structure somewhere with random names

Ah ok, this was the missing piece to my understanding.  So basically each file would be given a hashed name of say 10 - 12 characters, and store that hash along with the filename given by the user in the db.  And I would have a UNIQUE constraint on that column.

I don't know why that didn't cross my mind, but that answers my question.  Thanks Kicken!

[kicken]

1 hour ago, Zane said:

So basically each file would be given a hashed name of say 10 - 12 characters,

Yea, I usually just do something like this:

$storageDir = '/wherever/you/want';
do {
	$name = bin2hex(random_bytes(16));
} while (file_exists($storageDir.'/'.$name));

The loop probably isn't really necessary, the chance of a collision is statistically insignificant I think, but I throw it in anyway just in case.

I'll usually create a few sub-directories as well rather than literally have everything in one directory.  Take the first few characters of the name and make a sub-directory based on that.  For example, if the result were c6c1ce8c2bcf000daea561cbcca4a671 then I'd end up saving the file to: /wherever/you/want/c6/c1/c6c1ce8c2bcf000daea561cbcca4a671.  Keeps the directory sizes more manageable just in case you need to manually browse to some file.