No speed difference found when running exec(scp) in the background

[tirengarfio]

Hi,

I'm trying to copy asyncronously 10 70 MB video files.

 

exec("scp -o StrictHostKeyChecking=accept-new -i /var/keys/devDevices_rsa MarTianez1.mp4 awong@10.1.1.16:/tmp/test1 2>&1 > out.log", $output, $exitCode);
...
exec("scp -o StrictHostKeyChecking=accept-new -i /var/keys/devDevices_rsa MarTianez10.mp4 awong@10.1.1.16:/tmp/test10 2>&1 > out.log", $output, $exitCode);

 

But I cannot see any difference if I remove `2>&1 > out.log`'s from the scp commads.

 

PHP 8.1, Apache2, Ubuntu 22

Edited September 22, 2023 by tirengarfio

[requinix]

Do you know what those parts do?

[tirengarfio]

What do you mean with "parts"?

[requinix]

8 hours ago, tirengarfio said:

But I cannot see any difference if I remove `2>&1 > out.log`'s from the scp commads.

Those. The things you removed. Those parts.

[kicken]

Do you want to just kick off some copy commands and move on, or do you want your script to wait for them to finish and verify the copy was successful by checking the exit code?

 

[tirengarfio]

6 hours ago, requinix said:

Those. The things you removed. Those parts.

Those parts should put the scp command in the background so the exec()'s are executed before the scp's finish.

Edited September 23, 2023 by tirengarfio

[tirengarfio]

6 hours ago, kicken said:

Do you want to just kick off some copy commands and move on, or do you want your script to wait for them to finish and verify the copy was successful by checking the exit code?

 

I want to execute the exec("scp..") commands without waiting for the scp's to finish, what is called usually asynchronously.

[kicken]

The answer you found on stack overflow is incomplete.  In addition to redirecting the output, you need to actually put the operation into the background by including an & at the end.

exec("scp -o StrictHostKeyChecking=accept-new -i /var/keys/devDevices_rsa MarTianez1.mp4 awong@10.1.1.16:/tmp/test1 2>&1 > out.log &", $output, $exitCode);

This will just kick off the process and move on.  You won't have any way in your code to determine if the process completes successfully or not.  You'd only be able to detect if there is an error in the shell that prevented the command being run (syntax error, exec failure, etc).

If you want to be able to verify the copy is complete, you need to use something more complex than exec, which is the proc_open and related functions.  These let you run a command asynchronously while monitoring it's progress.  You can handle this easier by making use of the symfony/process package to execute your commands.  If you don't want to use a library, you can check my article on fibers for an example of how to run multiple processes asynchronously using proc_open.