PharData creating multiple tars creates many entries in /tmp for runtime duration

[iarp]

I have the following code that creates a tar file and then compresses it into a .tgz file.

$a = new PharData($tar_file);
$a->buildFromDirectory($source);
$a->compress(Phar::GZ, 'tgz');
@unlink($tar_file);

The script backups a number of different folders depending on time, day..etc. 

 

I can understand why PharData is creating the temporary file in /tmp/ and they all are named things like ./php8VMsXV ./phpaZsDDn and they are slightly larger than the uncompressed .tar file, so I'm guessing that PharData is using that as a temporary unnamed tar file, copies the contents into a properly named tar file and then compresses the .tar into the .tgz as I've specified above.

 

The issue is that PharData doesn't delete those /tmp/ files until the script finishes, it leaves them all in /tmp/ and then deletes at the end of script execution. Sometimes I have 35 directories that need to be archived and then uploaded to amazon. This is killing hard drive space on the cloud machines we have. We may have 20gb free but having 35 tmp files takes up the entire hard drive space and then all the scripts start failing.

 

I've tried unset($a) just after the compress and a couple other things that I've forgotten now. I can only find one thing online that seems to mention a potential issue with it, https://bugs.php.net/bug.php?id=70417

 

Any ideas?

[requinix]

Are you destroying $a when you're done with the compression? Try that.

 

If that doesn't help, destroy it and then call Phar::unlinkArchive.

[iarp]

That's what I'm trying to figure out, unlinkArchive only gets rid of the first .tar that it creates.

 

Both unset($a) and $a = null; do nothing.

[requinix]

Alright, I think I see a workaround: don't use Phar for the compression. It doesn't seem to clean up temporary files if you've enabled compression? Wouldn't expect it to use temp files in the first place... Instead use it to write out the tar archive and compress it yourself manually, such as by using one of:

`gzip {$tar_file}`; // creates $tar_file.gz

$in = fopen($tar_file, "rb");
$out = fopen("compress.zlib://{$tar_file}.gz", "wb"); // name.tar.gz; use a combination of dirname and basename to get just name.tgz
stream_copy_to_stream($in, $out);
fclose($out);
fclose($in);

[iarp]

It would have to store it somewhere, memory isn't a good location when it could be a very large archive by the time it's done.

 

I think I'm going to have to find another way without PharData. It's not the compression it's PharData itself. If I remove the compress call, it's still creating and holding onto the file in tmp.

 

Originally I started by calling exec to 'tar czvf $tgz_file $source' but I couldn't properly get the status or any error messages back. One error I'm constantly fighting is file paths that are too long. Some of these archive are failing because some file paths are 100+ characters which PharData craps out at 101+.

 

I've temporarily modified my script so that it exec's back onto itself and through argv it receives the folder that it needs to work on and does it thing, then when that script finishes since it was a standalone php call it clears tmp.

Edited March 31, 2016 by iarp

[iarp]

I've given up on PharData. It hasn't been a good experience, I'm back to using exec and tar czf and checking $results !== 0 for errors.

 

Issues so far were 100 filename character limitation, the tmp folder piling up during script execution, as well as other annoyances I don't think I really care for anymore.

# Compression
exec("tar czf $tgz_file -C $source .", $output, $results);

# Decompression
exec("tar xzf $tgz_file -C $output_folder", $output, $results);

Thanks for your help.

Edited March 31, 2016 by iarp