fetching data through curl

[abdulsamad]

OK guys i am back with my second question. And sorry for my 'bad language' in my 1st post i have already apologized to the mod.

Anyways here is the problem

i created a curl function to connect to the url i give here is the code

<?php  
  function curl_file_get_contents($url) {
      $ch = curl_init();
      curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
      curl_setopt($ch,CURLOPT_URL,$url); // the url of the site
      curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0); //needed for ssl (httpS://)
      $ret = curl_exec($ch);
      curl_close($ch);
      return($ret);
  }
?>

i want it to connect to the url i give for example

http://rapidshare.com/files/141992010/Patche_Lestrenge_up_for_www.DarkWarez.pl.part01.rar 

and fetch the server id frm there and make the link lyk

http://login:password@serverid.rapidshare.com/files/141992010/Patche_Lestrenge_up_for_www.DarkWarez.pl.part01.rar 

so how can i fetch the server id is there something wrong with my curl function?

[ghostdog74]

you should be using your own id , so you may want to specify your curl cookie options. eg cookiejar etc..

allow you may want to set followlocation to 1

 

[abdulsamad]

i don't follow you..

i wanna know how to get the server id from there? cauz when i manually open the link nd view its source it has a code like this at some point <form id"ff" action="rs355.rapidshare....." i wanna fetch that url so how to do it?

[thebadbad]

Since the site isn't https, why use it in the cURL function? Does it even work? Here's how to fetch the server id you're talking about:

 

<?php
$file = curl_file_get_contents('http://rapidshare.com/files/141992010/Patche_Lestrenge_up_for_www.DarkWarez.pl.part01.rar');
preg_match('~id="ff" action="http://([^.]+)~i', $file, $matches);
$server_id = $matches[1];

//build new URL
$login = 'login';
$password = 'password';
$new_url = "http://$login:$password@$server_id.rapidshare.com/files/141992010/Patche_Lestrenge_up_for_www.DarkWarez.pl.part01.rar";
?>

[abdulsamad]

well it says called to undefined function in line no.2 ...

[thebadbad]

You remembered to include the function in the script, right? Then cURL isn't installed/enabled on your server. You can use file_get_contents() instead, if you have allow_url_fopen set to 'On' in php.ini. Or enable cURL, if possible.

[abdulsamad]

WEll i uploaded it on a server with curl enabled but its giving now output just a blank page

http://noobsey.blackapplehost.com/Grabber/fetch.php

this is the url of the page no result

[thebadbad]

Look at the script. It's not supposed to output anything.

[abdulsamad]

Well i am noob in php but as far as i have understood it should give output after i use the echo command... But its not giving i used echo like this

echo $new_url;

am i doing it wrong?

[thebadbad]

That should work.

 

Looks like the host is interfering with the script - it says "Free Hosting by BlackAppleHost.com" on the page. Hopefully that's just added to the script output. And I guess error reporting is turned off?

 

Putting error_reporting(E_ALL); at the top of the script should turn on all error reports and warnings. There must be an error since there's no output.

[abdulsamad]

Well buddy blackapplehost.com is just the advertising of the host on which i have hosted the file.

Have you tested the script urself??? nd in the script we have just used curl but not used curl_init nd curl_exec commands ... shouldn't we use them every time we use curl?

[thebadbad]

in the script we have just used curl but not used curl_init nd curl_exec commands

 

When we call the curl_file_get_contents() user function, the curl commands are run (if you specified the function in the script, that is). You can use this function instead:

 

<?php
function store_page($url) {
if (function_exists('curl_init')) {
	$c = curl_init();
	curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
	curl_setopt($c, CURLOPT_URL, $url);
	$contents = curl_exec($c);
	curl_close($c);
	return $contents;
} elseif (ini_get('allow_url_fopen') == 1 && $contents = @file_get_contents($url)) {
	return $contents;
} else {
	die('Unable to scrape page. Be sure to have cURL installed or to have \'allow_url_fopen\' set to \'On\' in php.ini.');
}
}
?>

 

It will use cURL if installed, else file_get_contents(). Will output an error message if both fails. I just tested the script; it works. Only thing changed is the above function:

 

<?php
function store_page($url) {
if (function_exists('curl_init')) {
	$c = curl_init();
	curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
	curl_setopt($c, CURLOPT_URL, $url);
	$contents = curl_exec($c);
	curl_close($c);
	return $contents;
} elseif (ini_get('allow_url_fopen') == 1 && $contents = @file_get_contents($url)) {
	return $contents;
} else {
	die('Unable to crawl page. Be sure to have cURL installed or to have \'allow_url_fopen\' set to \'On\' in php.ini.');
}
}
$file = store_page('http://rapidshare.com/files/141992010/Patche_Lestrenge_up_for_www.DarkWarez.pl.part01.rar');
preg_match('~id="ff" action="http://([^.]+)~i', $file, $matches);
$server_id = $matches[1];

//build new URL
$login = 'login';
$password = 'password';
$new_url = "http://$login:$password@$server_id.rapidshare.com/files/141992010/Patche_Lestrenge_up_for_www.DarkWarez.pl.part01.rar";
echo $new_url;
?>

 

Output:

http://login:password@rs479.rapidshare.com/files/141992010/Patche_Lestrenge_up_for_www.DarkWarez.pl.part01.rar

[abdulsamad]

wow...!!!

you guyz really are good at php i m still a noob

[abdulsamad]

Can somebody please make it more flexible? I tried but was not able to do it

Here is the html page and 2 posts before is the php coding can somebody make it like i enter the pure rs links and it changes them into the list of ones with serverid in them??

<html>
<body>
<center><img src=rslogo.gif></center>
<form name=form1 action=linkready.php method=post>
Username: <input type=text name="us" />
Password: <input type=text name="us" />
Your RS LINKS:<br><textarea name=rsurls cols=100 rows=30></textarea><br><br>
<input type=submit value=Submit>
<input type=reset value=Reset>
</center>
</form>
</body>
</html>

 

[thebadbad]

Check it out below. I renamed the password input field name to "pw":

 

<?php
if ($_POST) {
function store_page($url) {
   if (function_exists('curl_init')) {
      $c = curl_init();
      curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
      curl_setopt($c, CURLOPT_URL, $url);
      $contents = curl_exec($c);
      curl_close($c);
      return $contents;
   } elseif (ini_get('allow_url_fopen') == 1 && $contents = @file_get_contents($url)) {
      return $contents;
   } else {
      die('Unable to crawl page. Be sure to have cURL installed or to have \'allow_url_fopen\' set to \'On\' in php.ini.');
   }
}

//retrieve URLs from textarea
$urls = str_replace(array("\r\n", "\r", "\n"), '<br />', $_POST['rsurls']);
$urls = explode('<br />', $urls);
//remove empty lines
$count = count($urls);
for ($i = 0; $i < $count; $i++) {
	if (trim($urls[$i]) == '') {
		unset($urls[$i]);
	}
}

//build new URLs
$new_urls = array();
foreach ($urls as $url) {
	$file = store_page($url);
	preg_match('~id="ff" action="http://([^.]+)~i', $file, $matches);
	$server_id = $matches[1];
	$nohttp = substr($url, 7);
	$new_urls[] = "http://{$_POST['us']}:{$_POST['pw']}@$server_id.$nohttp";
}

//list new URLs
foreach ($new_urls as $url) {
	echo "$url<br />";
}
} else {
?>
<html>
<body>
<center><img src=rslogo.gif></center>
<form name=form1 action=linkready.php method=post>
Username: <input type=text name="us" />
Password: <input type=text name="pw" />
Your RS LINKS:<br><textarea name=rsurls cols=100 rows=30></textarea><br><br>
<input type=submit value=Submit>
<input type=reset value=Reset>
</center>
</form>
</body>
</html>
<?php
}
?>

[thebadbad]

You may want to add

 

ini_set('user_agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; da; rv:1.9) Gecko/2008052906 Firefox/3.0');

 

at the top of the script (but inside the PHP tags), when grabbing the content of 'external' pages. If not set, the user agent is "PHP", and some sites block such request, since they know they are automated.

[abdulsamad]

THx a zillion dude you have made my work so very easy

i owe you one...

[thebadbad]

No problem