Batch Rename Issue

[monkeytooth]

Hey everyone, whats up? I have a slight issue with something I am working with and what I am trying to do is correct a massive mistake. We have 450,000+ images stored in a single directory on a remote server. We do have just about full access to the server, but despite that fact I'm sure that just about everyone already knows or can figure out what the possible issue is with this is. So needless to say I am attempting to build a script that can run through the directory of 450,000+ images and using rename() to move x amount of files to x directory, when x files is reached make a new directory and repeat til the process is complete. I am figuring or much more so guessing rather that 15,000 files per directory would be optimal for the over all cause.

 

Key issue. Due to the sheer size of the directory my scripts keep timing out. I need to figure out a way around this. It seems to be a memory limit issue (which we don't have control over unfortunately).

 

What I have attempted to put together at the moment is pure concept of the batch without actually running rename. Im using echo currently to output the concept to see if its going to work. I figure if I can get something working that is printing out in front of me switching from echo to rename later should work equally just as well.

 

What I have done so far

<?php
$directory = "ibook/Full_Images_1/";
$filescounted = count(glob("" . $directory . "*.jpg"));
echo $filescounted."<br /><br /><br />";
$fileCount = 0;
$dirtoDrop = "ibook/CR"
$dirCount = "100";
$dir = opendir($directory); 
while ($file = readdir($dir)) { 
  if (eregi("\.jpg",$file)) {
if($fileCount <= 5000){$dirCount = $dirCount;}else{$fileCount = 0; ++$dirCount; echo "<br /><br /><br />";}

echo '<strong>'.$dirtoDrop.$dirCount.'/</strong>'.$file.'<br />';
    $fileCount++;
  }
}

?>

 

this was working for a moment but something I did, made it stop, I'm not sure what though. I guess what I am doing here is looking to know if I am able to do this, and see if theres any minds out there greater than mind that can help me achieve this. If worse comes to worst I can do this on a local machine setting up a WAMP and then re-uploading all the images but I would like to avoid that if possible as its several gigs worth of files and as stated several hundred thousands of files. But I also still have a similar issue with the directory sizes. Only difference is I have sub directories of the images already but they are broken down into 65,000 files or so per directory. I want to try and trim it down to about 15,000 files per directory

 

One other thing I want to mention is I am keeping the actual file names the same, just using rename() to move them from one directory to another.

[Mchl]

Run script from console instead of from browser, and it will not timeout. If you absolutely need to run it from browser use set_time_limit

Also, echoing is slow, possibly slower than actually moving files.

 

Do you get any errors from this script?

[monkeytooth]

On the remote server I have no access to the CLI, locally I am not getting any errors, just nothingness..

[Mchl]

Do you have error reporting enabled?

[monkeytooth]

I believe so, I get general errors when i purposefully leave out a ; or a } or try to break the code..