Multiple remote upload server to server help

[amirf3131]

I want to make a small script allowing to upload multiple files from one server to another. The problem is when I add more than 100 urls it corrupts the files.

Any way to do it more clean so files should not be corrupted.

Here is the script:

// Check if form has been submitted

if(@$_POST['submit']){

ini_set("max_execution_time", 0);  // no time-outs!

ignore_user_abort(true);   // Continue downloading even after user closes the browser.

// URLS -- One on each line

$URL = $_POST['url'];

// Relative path to Save downloaded images

// Default is "downloads"

// Make sure that it is writable (chmoded correctly)

$folder = $_POST['folder']; 

// Check if User has provided a local folder

if (!$folder || !isset($folder)){

// Generate error if left blank by user.

die ("Please specify local folder name");

}

// Split all URLS into an array

$urls = split("\n", $URL);

// Remove Carriage Returns (useful for Windows-based browsers)

$urls = str_replace("\r", "", $urls);

$mh = curl_multi_init();

foreach ($urls as $i => $url) {

$path = pathinfo($url);

$g=$folder . "/" . $path["basename"] ;

// Check if file already exists on local folder.

if(file_exists($g)){

// If exists, delete the file so it always contains the latest update. 

unlink($g) or die("Unable to delete existing '$g'!");

}

// Update the user of what's going on

echo "$i) Downloading: from <b>$url</b> to <a href=\"$g\"><b>$g</b></a><br />";

if(!is_file($g)){

$conn[$i]=curl_init($url);

$fp[$i]=fopen ($g, "w");

curl_setopt ($conn[$i], CURLOPT_FILE, $fp[$i]);

curl_setopt ($conn[$i], CURLOPT_HEADER ,0);

// curl_setopt($conn[$i],CURLOPT_CONNECTTIMEOUT,1000);

curl_multi_add_handle ($mh,$conn[$i]);

}

}

do {

$n=curl_multi_exec($mh,$active);

}

while ($active);

foreach ($urls as $i => $url) {

curl_multi_remove_handle($mh,$conn[$i]);

curl_close($conn[$i]);

fclose ($fp[$i]);

}

curl_multi_close($mh);

} // task closed

?>

<br />

<br />

<fieldset>

<legend>

<label for="url">Server to Server Upload Script</label>

</legend>

<form method=POST>

<label for="url">Insert Files URL, One Per Line: </label><br />

<textarea rows=15 cols=75 id="url" name="url"><?= $URL ?></textarea><br />

<label for="folder">Folder Name: </label><input type=text id="folder" name="folder" value="uploads"/>

<input type=submit name="submit" value="Start Uploading Files!" />

</form>

</fieldset>

[requinix]

What does "corrupt" mean? How are the files corrupted? What is wrong with them?

[amirf3131]

What does "corrupt" mean? How are the files corrupted? What is wrong with them?

there is just file name and size is zero i think this is happens just because too many request sent at a time and i want to limit it untill first once is downloaded.?

Edited November 7, 2016 by amirf3131

[benanamen]

Why don't you tell us more in detail exactly what you are doing. I believe this is an XY problem.

[Jacques1]

Blindly adding a new connection for every URL is counter-productive. Find the sweet spot for the number of simultaneous connections and process the URLs groupwise (e. g. 10 at a time).

 

You also need a concept for handling duplicate filenames. Right now, you just throw everything into one folder. You either have to create subdirectories or make the filenames unique. By the way, the PHP path functions are for paths. You have to apply them to the path of the URL, not the entire URL.

 

Last but not least, you should fix your terminology. Right now, you can't seem to decide whether those are “uploads” or “downloads”. They're downloads. You fetch data from remote servers (uploading means sending data).

[amirf3131]

Why don't you tell us more in detail exactly what you are doing. I believe this is an XY problem.

 

the clean word is i just want to share multiple files from one server to another any way to do it more clean?