Process to Scan thousands of files

[Psycho]

I am working on a project to make a web interface for my MP3 collection. It will include functionality to read the tags from the MP3 files and save them to a database. But, I'm at a loss as to how I should set up the scanning process since it will be doing a large number of files. I will most likely have the ability to scan newly added albums individually as they are added, but I will also need to have a full scan process to account for files that are moved or have had the tags changed.

 

Each album has been ripped to a separate folder, so there are a few hundred folders and thousands of files. Timing out of the script will be an issue as well.

 

I was thinking that utilizing AJAX would be a possible solution so I could have one request process a folder, when that completes a new request would process the next folder, etc. That would prevent timeout issues. But, how would I keep track between JS and PHP as to which folder is next? I was thinking of havng PHP do an initial scan of just the folders and then passing that back to the Javascript or saving to a text file. Then using that to monitor the next folder to process. But, I'm wondering if there is a more elegant solution.

 

Any ideas or solutions are welcome.

[effigy]

If Perl is available, Apache::MP3 may be promising.

[Psycho]

Thanks, but that's appears to be a "complete" solution for accessing your MP3's. I'm building my own interface and am comfortable with all the functionality except for the process of scanning a large number of files.

 

FYI: There's a really nice free PHP app for this called Jinzora which I want to use, but it seems to always have several areas that are broken (installed different versions on several different machines/environments). So instead of reengineering someone elses code I decided I would build my own. http://en.jinzora.com/

[discomatt]

Sounds liek a good nightly cron to me. Simply call this

 

set_time_limit(0)

 

at the start, and it won't time out. Even then, PHP is fairly fast at iterating through files.. Unless it takes some serious power to pull the ID3 tags, i wouldn't worry about iterating through hundreds of folders once a night. I highly doubt it'd take more than 30 seconds anyway

[writer]

Unless it takes some serious power to pull the ID3 tags, i wouldn't worry about iterating through hundreds of folders once a night.

 

Agreed. I think it would depend on how often you're updating.

 

My Quicksilver mac app scans my computer every 10 minutes, which would be tricky, but I doubt you'll have to scan that often.

[GingerRobot]

Out of interest, how are you grabbing the ID3 tags?

[Psycho]

Out of interest, how are you grabbing the ID3 tags?

 

This works nicely:

getID3()

[Psycho]

Sounds liek a good nightly cron to me. Simply call this

 

set_time_limit(0)

 

This is going to be run on a Windows server. Perhaps a scheduled task?

 

EDIT: A little searching answered my own question (and it does involved scheduled tasks) Configuring cron jobs on Windows

[GingerRobot]

Out of interest, how are you grabbing the ID3 tags?

 

This works nicely:

getID3()

 

Cheers. Shame most of my library is in WMA which appears unsupported.

 

Edit: Actually, it can read WMA. Excellent - this could come in handy.

[Psycho]

Shame most of my library is in WMA which appears unsupported.

 

I think WMA is supported (it just doesn't stand out on that page). Under "Audio-Video formats" it states:

 

ASF (ASF, Windows Media Audio, Windows Media Video)