Jump to content

designing a social network - not sure on a couple of points


spiderwell

Recommended Posts

Hi All,

 

I have recently started a new role and its just me as developer, and the site is a social network, not a facebook rival, i might add :).

However when it comes to the design, i was not having much luck researching online for a few areas, and hope that some of you might be able to point me in the right direction of design tutorials or just articles about such subjects.

 

Uploads

Obviously this is the one thing all social networks need, and I am wondering the best way to go about storage. I was planning on using the filesystem for storing physical files such as images/pdfs, rather than put in the database (text entered in browser will be stored in this way). Is there a best practicise for storage techniques, I am thinking of having a folder with the same name as the user_id for all the uploads to go into, and then if i resize any images, they will be generated on the fly and stored in a cache folder. Would this be the correct way to do this, as I cant imagine stuffing it all into 1 folder makes any sense. Perhaps i could take user folder to have folders for each upload type images/pdfs/doc etc.

 

Connections

Now this is the fun bit, I again have been researching but not really found any articles that give me the bones and meat of what to do here from a database table structure view or just concept design. My first thoughts are a table for friends which is just a look up of user ids essentially, that would be 3 columns, id PK col, user_id, friend_id , perhaps a date column, and confirm friend column too.

 

At this stage these are the only points that I am trying to work out, I am not looking for a coded answer, I like to be pointed in the right direction and then do the work myself, so anything that might be useful, please let me know!

 

Many Thanks in advance

 

Nick

Link to comment
Share on other sites

Uploads

...

 

Splitting them up by UserID would work ok.  You may consider splitting up the user's directories also if you expect there may be a lot of users.  An alternative to splitting things by user id would be to just rename the file using either a random name, or maybe an auto-increment id of the row where the file details are stored. 

 

To split things up, one thing I do whenever I expect there may be a lot of files is divide them into sub directories based on the first couple letters of the name.  For example if I wanted to store the file 1928_somefile.jpg it would be stored into $UPLOADDIR/1/9/1928_somefile.jpg.

 

Connections

...

 

Basically a table like you mentioned is all you need to setup the connection.  You don't really need a separate auto-increment PK column though, just set your PK to be (userId, friendId). The combination should be unique and would be enough to identify the relationship in the table.

 

Link to comment
Share on other sites

thanks for the feedback, i guess thats a composite pk? i think thats the phrase, where there is a PK from combination of 2 columns, so you cant end up being friends twice.

 

would your example of folders and files mean that i might end up with many folders/paths only leading to 1 file in a folder?

would that not lead to having twice as many folders as there are files?, and if so I am not sure what implication this would have on performance, if any.

Link to comment
Share on other sites

would your example of folders and files mean that i might end up with many folders/paths only leading to 1 file in a folder?

would that not lead to having twice as many folders as there are files?, and if so I am not sure what implication this would have on performance, if any.

 

No, rather than ending up with one giant directory full of files (or directories) they get spread out over multiple directories.  If you just stuck it all into one directory and for some reason ever needed to go into that directory to do something you may have problems with things not handling a large number of files very well. 

 

If you want to have each user have their own folder, then you probably don't need to split the files up like that, just store them into the folder.  You might consider splitting the user's folders up that way though to prevent having a bunch of users folders in the same parent directory.  Such as maybe: Uploads/users/3/3456/somefile.jpg where 3456 is the user's ID#

 

 

Link to comment
Share on other sites

No, rather than ending up with one giant directory full of files (or directories) they get spread out over multiple directories. If you just stuck it all into one directory and for some reason ever needed to go into that directory to do something you may have problems with things not handling a large number of files very well.

To expand on this even more; there is a limitation on the number of "nodes" you can have in a certain directory. This number is much greater in ext4 filesystems than ext3, but still something you never want to reach.

Link to comment
Share on other sites

this is what i have come up with, its does the splitting of up of a userid into each character and turns that into a path, and then adds a folder which is named from the user id

for instance user with id 12345 gets a path of /upload/1/2/3/4/5/12345.

define('UPLOADS_FOLDER', './upload/');


function make_user_upload_path($userid)
   {
       //create a path for the user for uploading
       if(strlen($userid) == 1)
       {
           if(!file_exists(UPLOADS_FOLDER . $userid)) {
               mkdir(UPLOADS_FOLDER . $userid,0777,true);
           }
           return UPLOADS_FOLDER . $userid . '/';//add trailing slash
       }
       else
       {
           $arrTemp = str_split($userid);
           $newPath = implode('/',$arrTemp);
           if(!file_exists(UPLOADS_FOLDER . $newPath . '/' . $userid)) {
               mkdir(UPLOADS_FOLDER . $newPath . '/' . $userid ,0777,true);
           }
           return UPLOADS_FOLDER . $newPath . '/' . $userid . '/';//add trailing slash
       }

   }

i think maybe i am splitting it up too much, maybe only split upto 3 or 4 characters would be enough?

Edited by spiderwell
Link to comment
Share on other sites

You don't want to go too deep either, because that will greatly affect performance as well.

 

A while back I posted a link to a discussion that was on this specific topic; Storage design for a large number of files on a file systems. I'll see if I can't find it again, and give you the link.

In the meantime, I do recommend searching the forum for it as well; You might find it before I do. ;)

 

Added:

Hah! I found it. :)

Edited by Christian F.
Link to comment
Share on other sites

having looked at that article, and links to others in it, it seems that i do not want to go to deep, but i think i wont with the system i came up with, if i had 90 million users, the structure would be 8 folders deep. but i am a bit niave on things like this, perhaps that is too deep already?

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.