Link checking

[thefollower]

Hey,

 

I use a file host to store alot of my back ups but the links die from time to time. So i thought i should try and make my own link checker - but when i google for scripts i just find websites which offer a link checker... rather than a guide on how to make them..

 

Any one got any idea where to begin with link checkers on sites like Megaupload/Rapidshare ?

[teamatomic]

I would use Curl and just get the response.

$ch = curl_init('http://yoururl/');

curl_setopt($ch, CURLOPT_HEADER, 1);

$c = curl_exec($ch);

echo curl_getinfo($ch, CURLINFO_HTTP_CODE);

 

Here's a list of codes

http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

 

 

HTH

Teamatomic

[thefollower]

Gave it ago and it loads megaupload inside my site i guess its meant to! But i don't get how i can just check the link is live from that?

 

I was thinking more like just it returning "Live" "Unavailable" or "Dead". Although looks like you got the idea - but i need to some how return just response rather than load the page - if you get me ?

[teamatomic]

$url='http://www.google.com';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
preg_match('*HTTP\/1\.1\s([0-9]{3})*is', $result, $status);
echo "$status[1]";

 

 

HTH

Teamatomic

[thefollower]

Hmm no this isn't working - i tested it with a dead link and it still returned value 200 which according to wikipedia means its OK but it's not cos its deffinately a dead link.

[teamatomic]

What do you mean a "dead link"? It does work.

try

google.com

and

www.google.com

and

www.google.com/this_is_dead.html

you get, in order

200,301,404

 

So where is it not working?

 

HTH

Teamatomic

[merylvingien]

I am still trying to learn how to check if a link is present within a web page, if anyone has any tips?

[premiso]

I am still trying to learn how to check if a link is present within a web page, if anyone has any tips?

 

This topic is a bit different. If you would like someone to address this issue, please create your own topic for it. Thanks.

 

As to the original OP, if you have not solved your issue, please post the link you are trying to test if it is dead or not.

[thebadbad]

You simply have to find a way do distinguish a page with a 'dead' file from a page with a live file. A Rapidshare page e.g. contains the header <h1>FILE DOWNLOAD</h1> when the file is live. You can simply use strpos() to check if that's present in the source code:

 

<?php
$source = file_get_contents('http://rapidshare.com/files/326980450/TOIOU_NIN_AVOTT_GIFT_1080p.part01.rar');
if (strpos($source, '<h1>FILE DOWNLOAD</h1>') !== false) {
//file is live
}
?>

[thefollower]

What do you mean a "dead link"? It does work.

try

google.com

and

www.google.com

and

www.google.com/this_is_dead.html

you get, in order

200,301,404

 

So where is it not working?

 

HTH

Teamatomic

 

Say a megaupload file no longer exists on megaupload's server it still seems to return its live - for example:

 

Example try it with this:

 

http://www.megaupload.com/?d=62LER87 < this is a dead link the file for that link is not found on megaupload's server but it returns live yet the upload is still dead.

[teamatomic]

Thats simple. Use the first code that returns a full page and parse out the "invalid link" text.

I would try numerous dead links on the various sites you are going to check and put the strings into an array and then parse against the array.

 

 

HTH

Teamatomic

[oni-kun]

Yeah, the website in this case redirects you to a custom 404 page, so it'll appear as '200' anyway, since the client won't know the error code. As mentioned you can  search for the text 'Unfortunately, the link you have clicked is not available.' with strpos, and your answer will be solved!

[thefollower]

What variable does the text pass into ?

[oni-kun]

What variable does the text pass into ?

 

It's static text, Try this..

$sitecontents = file_get_contents('http://www.megaupload.com/?d=62LER87');
if (preg_match('/invalid link/i',$sitecontents)) {
   echo "File does not exist.";
} else {
   echo "File available.";
}

 

EDIT: fixed code a little

[teamatomic]

Just take this

echo curl_getinfo($ch, CURLINFO_HTTP_CODE);

and put it to a var

$result= curl_getinfo($ch, CURLINFO_HTTP_CODE);

 

This is from the first post of code that returned the whole page. The second set of code just returns the headers.

 

 

HTH

Teamatomic

 

[thefollower]

I tried this:

 

The second link in $Array is invalid but it returns valid.

<?php
$Array = 'http://www.megaupload.com/?d=ZD6ACN1J,http://www.megaupload.com/?d=ZACN1J';
$pieces = explode(",", $Array);
foreach ($pieces As $var){
Echo '<b>'.$var.'</b> - ';
if (preg_match('/Unfortunately, the link you have clicked is not available./i',$var)) {
   echo "<font color=red>File does not exist.</font>";
} else {
   echo "<font color=green>File available.</font>";
}
echo '<br>';
}
?>

[thebadbad]

You forgot to load the source code of the page.

 

<?php
$urls = array('http://www.megaupload.com/?d=ZD6ACN1J', 'http://www.megaupload.com/?d=ZACN1J');
foreach ($urls as $url) {
$source = file_get_contents($url);
echo "<strong>$url</strong>: ";
if (preg_match('~<input\b[^>]*>~i', $source)) {
	echo '<span style="color: green;">File is available.</span><br />';
} else {
	echo '<span style="color: red;">File does not exist.</span><br />';
}
}
?>

 

Megaupload serves a language specific error message, so I search for an input tag instead of the phrase in your code.

[thefollower]

http://i49.tinypic.com/9ut53c.jpg < see image

Still doesn't work, all bar one is invalid but they all come out invalid bar the first one.

 

I used:

 

<?php
$Array =  'http://www.megaupload.com/?d=62LER87B,
http://www.megaupload.com/?d=04L0TC2M,
    http://www.megaupload.com/?d=09P3EHZL,
http://www.megaupload.com/?d=ZACN1J,
    http://www.megaupload.com/?d=0C6NFC2Y,
    http://www.megaupload.com/?d=0GPWC3OJ,
    http://www.megaupload.com/?d=0LUAWPF0,
    http://www.megaupload.com/?d=0OOOXXJX,
    http://www.megaupload.com/?d=0R8JP2V7,
    http://www.megaupload.com/?d=441KFFEZ';

$pieces = explode(",", $Array);
foreach ($pieces As $var){
$source = file_get_contents($var);
   echo "<strong>$url</strong>: ";
   if (preg_match('~<input\b[^>]*>~i', $source)) {
      echo '<span style="color: green;">File is available.</span><br />';
   } else {
      echo '<span style="color: red;">File does not exist.</span><br />';
   }
}
?>

[thebadbad]

All of your URLs except the first one have whitespace in front of them. Either remove the whitespace or manually construct a proper array.

[thefollower]

All of your URLs except the first one have whitespace in front of them. Either remove the whitespace or manually construct a proper array.

 

Thanks works now!