Jump to content

Process to Scan thousands of files


Psycho

Recommended Posts

I am working on a project to make a web interface for my MP3 collection. It will include functionality to read the tags from the MP3 files and save them to a database. But, I'm at a loss as to how I should set up the scanning process since it will be doing a large number of files. I will most likely have the ability to scan newly added albums individually as they are added, but I will also need to have a full scan process to account for files that are moved or have had the tags changed.

 

Each album has been ripped to a separate folder, so there are a few hundred folders and thousands of files. Timing out of the script will be an issue as well.

 

I was thinking that utilizing AJAX would be a possible solution so I could have one request process a folder, when that completes a new request would process the next folder, etc. That would prevent timeout issues. But, how would I keep track between JS and PHP as to which folder is next? I was thinking of havng PHP do an initial scan of just the folders and then passing that back to the Javascript or saving to a text file. Then using that to monitor the next folder to process. But, I'm wondering if there is a more elegant solution.

 

Any ideas or solutions are welcome.

Link to comment
https://forums.phpfreaks.com/topic/100353-process-to-scan-thousands-of-files/
Share on other sites

Thanks, but that's appears to be a "complete" solution for accessing your MP3's. I'm building my own interface and am comfortable with all the functionality except for the process of scanning a large number of files.

 

FYI: There's a really nice free PHP app for this called Jinzora which I want to use, but it seems to always have several areas that are broken (installed different versions on several different machines/environments). So instead of reengineering someone elses code I decided I would build my own. http://en.jinzora.com/

Sounds liek a good nightly cron to me. Simply call this

 

set_time_limit(0)

 

at the start, and it won't time out. Even then, PHP is fairly fast at iterating through files.. Unless it takes some serious power to pull the ID3 tags, i wouldn't worry about iterating through hundreds of folders once a night. I highly doubt it'd take more than 30 seconds anyway

Unless it takes some serious power to pull the ID3 tags, i wouldn't worry about iterating through hundreds of folders once a night.

 

Agreed. I think it would depend on how often you're updating.

 

My Quicksilver mac app scans my computer every 10 minutes, which would be tricky, but I doubt you'll have to scan that often.

Sounds liek a good nightly cron to me. Simply call this

 

set_time_limit(0)

 

This is going to be run on a Windows server. Perhaps a scheduled task?

 

EDIT: A little searching answered my own question (and it does involved scheduled tasks) Configuring cron jobs on Windows

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.