download from website to my website?

[seany123]

Is it possible to download files from a website to my online websites root directory? reason I'm asking is because I have been downloading large video files to my computer which take about 20 minutes!! Then I use FTP to upload them to my site but it takes about 2-3 hours per video!! I was looking for a faster way! All help would be great!

[The Little Guy]

Untested:

 

<?php

$url = 'http://www.example.com/myVideo.avi';
$pi = pathinfo($url);
$ext = $pi['extension'];
$name = $pi['filename'];


// create a new cURL resource
$ch = curl_init();

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// grab URL and pass it to the browser
$opt = curl_exec($ch);

// close cURL resource, and free up system resources
curl_close($ch);

$saveFile = $name.'.'.$ext;

$handle = fopen($saveFile, 'rb');
fwrite($handle, $opt);
fclose($handle);
?>

[seany123]

Untested:

 

<?php

$url = 'http://www.example.com/myVideo.avi';
$pi = pathinfo($url);
$ext = $pi['extension'];
$name = $pi['filename'];


// create a new cURL resource
$ch = curl_init();

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// grab URL and pass it to the browser
$opt = curl_exec($ch);

// close cURL resource, and free up system resources
curl_close($ch);

$saveFile = $name.'.'.$ext;

$handle = fopen($saveFile, 'rb');
fwrite($handle, $opt);
fclose($handle);
?>

 

that looks very promising i didnt even realise php was capable of this... wheres the part where it saves the files to my website? also is there a way to choose where it can go?

 

also im getting this set of errors:

 

Warning: fopen(spnyenupqobm.avi) [function.fopen]: failed to open stream: No such file or directory on line 27

Warning: fwrite(): supplied argument is not a valid stream resource on line 28

Warning: fclose(): supplied argument is not a valid stream resource on line 29

[The Little Guy]

oops, change this:

$handle = fopen($saveFile, 'rb');

 

to this:

$handle = fopen($saveFile, 'w');

 

also, this line:

$saveFile = $name.'.'.$ext;

 

Is what you want to use to save. Right now it it uses the current directory of the php file.

 

This:

$saveFile = '../videos/'.$name.'.'.$ext;

 

Will save it in a directory above the current php file, and then into a videos directory at that level (the directory must exist)

[seany123]

that comes up in my root folder but its comes up at 0kbs.. im guessing it times out.

 

im guessing it would very fairly impossible to do a file like 100mb+

[The Little Guy]

Okay with a few mods, I ran this on my server and it worked fine:

 

<?php
set_time_limit(0);
$url = 'http://www.example.com/video.avi';
$pi = pathinfo($url);
$ext = $pi['extension'];
$name = $pi['filename'];


// create a new cURL resource
$ch = curl_init();

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// grab URL and pass it to the browser
$opt = curl_exec($ch);

// close cURL resource, and free up system resources
curl_close($ch);

$saveFile = $name.'.'.$ext;

$handle = fopen($saveFile, 'wb');
fwrite($handle, $opt);
fclose($handle);
?>

[seany123]

for some reason the file is reaching 345kbs and thats all.

[The Little Guy]

What is the url?

[seany123]

i have tried some different urls... all from stagevu.com

 

here is an example of one which i have tried this file is 581mb... but im, only able to dl 345kbs

 

http://n60.stagevu.com/v/56d446709c46bdaa998942ea0b7a2ff5/fwhpvmkgytlj.avi

[seany123]

anyone got any idea how to get the file completely downloaded??

[PaulRyan]

Well when accessing that URL you provided I recieve a 403 - Forbidden error.

 

So it would probably help if you had the correct URL...

 

Video URL: http://n60.stagevu.com/v/47e20980717da2108283a9c4bc905fc1/fwhpvmkgytlj.avi [Picture Perfect]

 

*Edited Post Below*

I have created a simple form and file to download files.

 

Filename: download.php

<?PHP if($_POST) { ?>

Filename: <?PHP echo $_POST['filename']; ?>
<script type="text/javascript">
<!--
  function doDownload() {
    if(window.XMLHttpRequest) {
      xmlhttp=new XMLHttpRequest();
    } else {
      xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
    }
    xmlhttp.onreadystatechange=function() {
      if(xmlhttp.readyState==4 && xmlhttp.status==200) {
        document.getElementById("downloadStatus").innerHTML=xmlhttp.responseText;
      }
    }
    xmlhttp.open("GET","file-download.php?url=<?PHP echo $_POST['url']; ?>&filename=<?PHP echo str_replace(' ','_',$_POST['filename']); ?>",true);
    xmlhttp.send();
  }

  setInterval('doDownload()',2500);
//-->
</script>

<div id="downloadStatus"></div>

<?PHP } else { ?>

<form method="POST" action="?">
File URL: <br><input type="text" name="url" value="" size="100"><br>
New name (with extension): <br><input type="text" name="filename" value="" size="50"><br>
<input type="submit" name="get" value="Get File">
</form>

<?PHP } ?>

 

Filename: file-download.php

<?PHP set_time_limit(0);

  $filename = str_replace(' ','_',$_GET['filename']);
  $fileURL  = $_GET['url'];

  $ch = curl_init($fileURL);
  curl_setopt($ch, CURLOPT_NOBODY, true);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_HEADER, true);
  $data = curl_exec($ch);
  curl_close($ch);

  $filesize = explode('Content-Length: ',$data);
  $filesize = explode("\n",$filesize[1]);
  $filesize = $filesize[0];

  //echo $filename,' - ';
  //echo $filesize,' - ',filesize($filename);

    if(!file_exists($filename)) {
      copy(trim($_GET['url']),$filename);
    } else if(file_exists($filename) && filesize($filename) == $filesize) {
      echo 'Downloaded'; exit;
    } else {
      echo 'Downloading "'.$_GET['filename'].'"<br>'.
           '<div style="padding: 5px; text-align: left;"><div style=" -moz-border-radius: 5px; margin: 3px; border: 1px solid #000; height: 15px; width: 150px;"><div style="width: '.number_format((filesize($filename)/$filesize)*100,2).'%; background: url(\'loader_background.png\') repeat-x center; height: 15px;"></div></div> ['.number_format((filesize($filename)/$filesize)*100,2).'%]</div>';
    }

  function dlTime($timestamp) {
    $minutes = floor($timestamp/60);
    $seconds = floor($timestamp-($minutes*60));
    $minEnd = $minutes == 1 ? 'min':'mins';
    $secEnd = $seconds == 1 ? 'sec':'secs';
    return $minutes.' '.$minEnd.' '.$seconds.' '.$secEnd;
  }

  $timePassed = (time()-filectime($filename));
  $sizeDone   = number_format((filesize($filename)/$filesize)*100,2);

  $timeLeft = ((100/$sizeDone)*$timePassed)-$timePassed;

  echo 'Estimated Time Left: '.dlTime($timeLeft).'<br>';

  echo 'Time Passed: '.dlTime($timePassed);
?>

Regards, PaulRyan.

[QuickOldCar]

Your script works good paulryan.

 

The url you did was also 403 - Forbidden error.

 

They most likely deleted it or they cycle file names/encrypt so people can't abuse their bandwidth.

 

I'm sure work fine many other sites, or would just need to get the latest url from the website, still saves from downloading and then uploading to the server.

[PaulRyan]

Thanks, it's something I've had for a little while for doing the same as you

 

Yeah, I've found out it changes every 15 minutes or so.

You could always make a function to load the StageVu Video URL [http://stagevu.com/video/fwhpvmkgytlj] parse the page to retrieve the file URL then whenever you need to download another movie it's just a case of inputting the StageVu Video URL.

 

Something you can maybes expand on, if you get stuck let me know and I'll see what I can do

 

Regards, PaulRyan.

[QuickOldCar]

Yeah that's the way would have to.

 

I think would have to make specific for each site though, they all have different methods.

 

I'd parse the url to know the site:

stagevu.com

using html dom parse all the href on the page:

then depending on which site use w/e pattern they use like the path, like stagevu uses

for stagevu the first file path is always v, but since they always have just one avi as a download easier to just search the end url's by filetype .avi

http://n65.stagevu.com/v/63980ee118e01d9980d0eacacc459ad2/jomxdgbvxnip.avi

then it can continue with the download script

 

all in all I think it's easier to just get the current links from the sites, lol

 

But here is a meatballed version that uses simplehtml dom

It requires simple_html_dom.php in same directory.

http://simplehtmldom.sourceforge.net/

 

I really didn't do checks or anything, just to see it work a few times.

 

So using urls such as http://stagevu.com/video/srpfmbeqxlwe instead of the avi download link in the form

 

 

file-download.php

<?PHP set_time_limit(0);

  $filename = str_replace(' ','_',$_GET['filename']);
  $fileURL  = $_GET['url'];
  
include('simple_html_dom.php');

function getHost2($url) {
            $parseUrl = parse_url(trim($url));
            return trim($parseUrl[host] ? $parseUrl[host] : array_shift(explode('/', $parseUrl[path], 2)));
        }

$url = mysql_real_escape_string($_GET['url']);

$getExtensions = array("avi","avi");

if (substr($url, 0, 4) != "http") {
$url = "http://$url";
}

$parsed_url = getHost2($url);

$http_parsed_host = "http://$parsed_url/";
$html = file_get_html($url);
?>
<base target="_blank">
<?php
foreach($html->find('a') as $element) 
$title = "$element->href";
$dom = new DOMDocument();
@$dom->loadHTML($html);


$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate("/html/body//a");

for ($i = 0; $i < $hrefs->length; $i++) {
$href = $hrefs->item($i);
$href_link = $href->getAttribute('href');

$parse_count = count("$http_parsed_host");
$substr_count = +7;


if (substr($href_link, 0, $substr_count) == "mailto:") {
$mail_link = $href_link;
$href_link = trim($mail_link,$href_link);

}


if (substr($href_link, 0, 1) == "/") {
$href_link = trim($href_link,"/");

}   

if (substr($href_link, 0, 2) == "//") {
$href_link = trim($href_link,"//");

} 

if (substr($href_link, 0, 3) == "///") {
$href_link = trim($href_link,"///");

}  


if ((substr($href_link, 0,  == "https://") OR (substr($href_link, 0, 12) == "https://www.") OR (substr($href_link, 0, 7) == "http://") OR (substr($href_link, 0, 11) == "http://www.") OR (substr($href_link, 0, 6) == "ftp://")  OR (substr($href_link, 0, 11) == "feed://www.") OR (substr($href_link, 0, 7) == "feed://")) {

         $final_href_link[] = $href_link;
  

} else {

  if (substr($href_link, 0, 1) != "/") {
$final_href_link[] = "$http_parsed_host$href_link";
}

}             
}
$links_array = array_unique($final_href_link);
sort($links_array);
foreach ($links_array as $links) {
if (in_array(end(explode(".",
            strtolower($links))),
            $getExtensions)) {

$fileURL = $links;
echo "Downloading: <a href='$fileURL'>$fileURL</a><br />";
}
//echo "$links<br />";
echo "<a href='$links'>$links</a><br />";
//echo "$title";
//$(‘title’).text($links);
}
echo "<a href='$mail_link'>$mail_link</a><br />";




  $ch = curl_init($fileURL);
  curl_setopt($ch, CURLOPT_NOBODY, true);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_HEADER, true);
  $data = curl_exec($ch);
  curl_close($ch);

  $filesize = explode('Content-Length: ',$data);
  $filesize = explode("\n",$filesize[1]);
  $filesize = $filesize[0];

  //echo $filename,' - ';
  //echo $filesize,' - ',filesize($filename);

    if(!file_exists($filename)) {
      copy(trim($_GET['url']),$filename);
    } else if(file_exists($filename) && filesize($filename) == $filesize) {
      echo 'Downloaded'; exit;
    } else {
      echo 'Downloading "'.$_GET['filename'].'"<br>'.
           '<div style="padding: 5px; text-align: left;"><div style=" -moz-border-radius: 5px; margin: 3px; border: 1px solid #000; height: 15px; width: 150px;"><div style="width: '.number_format((filesize($filename)/$filesize)*100,2).'%; background: url(\'loader_background.png\') repeat-x center; height: 15px;"></div></div> ['.number_format((filesize($filename)/$filesize)*100,2).'%]</div>';
    }

  function dlTime($timestamp) {
    $minutes = floor($timestamp/60);
    $seconds = floor($timestamp-($minutes*60));
    $minEnd = $minutes == 1 ? 'min':'mins';
    $secEnd = $seconds == 1 ? 'sec':'secs';
    return $minutes.' '.$minEnd.' '.$seconds.' '.$secEnd;
  }

  $timePassed = (time()-filectime($filename));
  $sizeDone   = number_format((filesize($filename)/$filesize)*100,2);

  $timeLeft = ((100/$sizeDone)*$timePassed)-$timePassed;

  echo 'Estimated Time Left: '.dlTime($timeLeft).'<br>';

  echo 'Time Passed: '.dlTime($timePassed);
?>

 

 

[PaulRyan]

I never knew of the Simple HTML DOM 

You learn something new everyday.

 

But your method seems long winded when you can just use explode() like the following...

    $fileURL = file_get_contents($_GET['url']);
    $fileURL = explode('<script type="text/javascript">',$fileURL);
    $fileURL = explode('\'',$fileURL[3]);
    $fileURL = trim($fileURL[1]);

 

The above will do what you have done in alot less code, not saying either method is the correct way of doing it though, all down to preference.

 

Easy to edit if and when they change the the coding on StageVu, I've tested a few different links and all work with my method

 

Regards, Paul.

[QuickOldCar]

I agree with your less code for extension, I just used my array one I use for a files upload script for allowed extensions. I was thinking on making it that depending on the extension and which domain was coming from to do different things with it.

Maybe a multi-downloader and checkboxes for all would like to download on the page and place them in different locations. Was just an idea, but I already have way too many ideas.

 

Here's my simple page parser

http://dynaindex.com/simple-parse

 

Or a more advanced one I did.

http://dynaindex.com/page-parser

 

Yeah simplehtml dom pretty cool, you can grab from any tag you want, also grab titles, I don't do titles actually because I found that not every link has a title, plus many sites use self ref so as can see in that part of my code I made many corrections for making them an actual link.

[seany123]

after typing the url and new filename i seem to just be left on download.php which the filename echoing and nothing else happening!

 

is it something im doing wrong?

 

just realised i have a error log file.

 

[10-Jan-2011 01:13:33] PHP Warning:  copy() [<a href='function.copy'>function.copy</a>]: Filename cannot be empty in file-download.php on line 21

[10-Jan-2011 01:13:33] PHP Warning:  Division by zero in file-download.php on line 38

[10-Jan-2011 01:13:33] PHP Warning:  Division by zero in file-download.php on line 40

 

[seany123]

still haven't been able to get it to work.