Link Checker + Time it took

jjmusicpro · October 6, 2007

Ok, so i looked around and didnt find anything on this, so here i am!

I have a table named activeurls, with 2 colums, url_id and url. I have around 300 records of urls...

examples of these would be www.joesplace.com/someplace.html etc..

I wanted to know how to run a script that will check the url to see if the page loads, or times out etc. plus the time it took to respond.

These can just be kicked out on the same page...

I dont know where to start....

Here is what i have, i dont know how to run each url through the loop, and get the time it took for it to load the page.

For now, i will only care about the pages not repsonding, or broken links for error codes.

ANy ideas?

<head></head>
<body>
<form method="post">
Check Url Status:<br>
<input type="submit" name="submit">
</form>
<?php
if($_SERVER['REQUEST_METHOD'] == "POST"){ 

$query = "SELECT * from activeurls";
$result2 = mysql_query($query); 

if($count==""){ 
echo "Done checking";
}else{ 
while($row = mysql_fetch_array($result, MYSQL_ASSOC)){ 
$url_id = $row['url_id'];
$url = $row['url'];
echo "URLID . $url_id . $url";

}
?>
</body>

MadTechie · October 6, 2007

you have already opened a thread here

i am also wondering what you had to PM instead of making public!!

so i think something fishy is going on here!

heckenschutze · October 6, 2007

Or phishy, get it phishy...

MasterACE14 · October 6, 2007

lol, phishy, fishy, similar yet different.

PM'ing someone about a question or problem you have isn't the way to ask a question, this is a PHP Forum for asking questions to a large crowd, not singling out the professionals to get a quick answer, and double posting isn't the right way to go about it either.

Regards ACE

jjmusicpro · October 6, 2007

I PM bluesky because i think this is a super tuff php thing to do, so i was going to give him a few bucks in paypal to help me get started.

MadTechie · October 6, 2007

did he solve it ?,

you could of still used that thread..

a few bucks.. humm maybe i would wait for it to be move to the freelance section...

MasterACE14 · October 6, 2007

yeah, sounds a bit like bribery lol

does this thread ring a bell? http://www.phpfreaks.com/forums/index.php/topic,162226.0.html

MadTechie · October 6, 2007

Okay heres a soultion, not 100% perfect but works well

<?php
$URLarray = array(
"http://uk2.php.net", 
"http://gfdghdsfghsdgfhsdghj.net", 
"http://www.phpfreaks.com/"	);

echo "<table>\n";
echo "  <tr>\n";
echo "      <td>\n";
echo "URL"; 
echo "      </td>\n";
echo "      <td>\n";
echo "Time"; 
echo "      </td>\n";
echo "  </tr>\n";

foreach($URLarray as $URL)
{
echo "  <tr>\n";
$mtime = microtime(); 
$mtime = explode(' ', $mtime); 
$mtime = $mtime[1] + $mtime[0]; 
$starttime = $mtime; 
if( url_exists($URL) )
{
	$data = file_get_contents($URL);
	unset($data);//dump	
	$mtime = microtime(); 
	$mtime = explode(" ", $mtime); 
	$mtime = $mtime[1] + $mtime[0]; 
	$endtime = $mtime; 
	$totaltime = ($endtime - $starttime); 
	$UrlResults[$URL] = array("Access" => true, "Time"=>$totaltime); 
	echo "      <td>\n";
	echo "$URL"; 
	echo "      </td>\n";
	echo "      <td>\n";
	echo "$totaltime seconds."; 
	echo "      </td>\n";
}else{
	$UrlResults[$URL] = array("Access" => false); 
	echo "      <td>\n";
	echo "$URL"; 
	echo "      </td>\n";
	echo "      <td>\n";
	echo "Failed"; 
	echo "      </td>\n";
}
echo "  </tr>\n";
flush();
}
echo "</table>\n";

//Results from loop (if you want)
//foreach($UrlResults as $URLs => $Tests)
//{
//	echo "$URLs: ";
//	if($Tests['Access'])
//	{
//		echo "Works access time:".$Tests['Time'];
//	}else{
//		echo "Failed";
//	}
//	echo "<br />";
//}


function url_exists($strURL)
{
    $resURL = curl_init();
    curl_setopt($resURL, CURLOPT_URL, $strURL);
curl_setopt($resURL, CURLOPT_BINARYTRANSFER, 1);
    curl_setopt($resURL, CURLOPT_HEADERFUNCTION, 'readHeader');
    curl_setopt($resURL, CURLOPT_FAILONERROR, 1);

    $x = curl_exec($resURL);

    $intReturnCode = curl_getinfo($resURL, CURLINFO_HTTP_CODE);
    curl_close ($resURL);

    if ($intReturnCode != 200 && $intReturnCode != 302 && $intReturnCode != 304) {
       return false;
    }else{
        return true ;
    }
} 
function readHeader($data)
{
return $data;
}
?>

that will be $15 please, Just kidding

the flaw is it downloads the html not the images,

of course you can remove the

		$data = file_get_contents($URL);
	unset($data);//dump

code to check the resolved rate instead of page download time, that will also work..

i hope this help

MasterACE14 · October 6, 2007

that will be $15 please, Just kidding

what currency? lol

MadTechie · October 6, 2007

well its should be UK Pounds but i accepts anything lol

jjmusicpro · October 6, 2007

Thanks MadTechie!

I tried running it, however it just sits there and thinks....

Is there a way to...if the site does not load within 8 seconds then count it as a 404 error?

Alos, is there a way for this to display the results as it does them?

Thanks again...

d.shankar · October 6, 2007

Yea you can do it with Timeout option.

curl_setopt($ch,CURLOPT_TIMEOUT,;

jjmusicpro · October 6, 2007

When i run the script, it just keeps going and going, never stops lol...

<head>Test Cheker 1.0v</head>
<body>

<?php
$URLarray = array(
"http://www.phpfreaks.com/"	);

echo "<table>\n";
echo "  <tr>\n";
echo "      <td>\n";
echo "URL"; 
echo "      </td>\n";
echo "      <td>\n";
echo "Time"; 
echo "      </td>\n";
echo "  </tr>\n";

foreach($URLarray as $URL)
{
echo "  <tr>\n";
$mtime = microtime(); 
$mtime = explode(' ', $mtime); 
$mtime = $mtime[1] + $mtime[0]; 
$starttime = $mtime; 
if( url_exists($URL) )
{
	$data = file_get_contents($URL);
	unset($data);//dump	
	$mtime = microtime(); 
	$mtime = explode(" ", $mtime); 
	$mtime = $mtime[1] + $mtime[0]; 
	$endtime = $mtime; 
	$totaltime = ($endtime - $starttime); 
	$UrlResults[$URL] = array("Access" => true, "Time"=>$totaltime); 
	echo "      <td>\n";
	echo "$URL"; 
	echo "      </td>\n";
	echo "      <td>\n";
	echo "$totaltime seconds."; 
	echo "      </td>\n";
}else{
	$UrlResults[$URL] = array("Access" => false); 
	echo "      <td>\n";
	echo "$URL"; 
	echo "      </td>\n";
	echo "      <td>\n";
	echo "Failed"; 
	echo "      </td>\n";
}
echo "  </tr>\n";
flush();
}
echo "</table>\n";

//Results from loop (if you want)
//foreach($UrlResults as $URLs => $Tests)
//{
//	echo "$URLs: ";
//	if($Tests['Access'])
//	{
//		echo "Works access time:".$Tests['Time'];
//	}else{
//		echo "Failed";
//	}
//	echo "<br />";
//}


function url_exists($strURL)
{
    $resURL = curl_init();
    curl_setopt($resURL, CURLOPT_URL, $strURL);
curl_setopt($resURL, CURLOPT_BINARYTRANSFER, 1);
    curl_setopt($resURL, CURLOPT_HEADERFUNCTION, 'readHeader');
    curl_setopt($resURL, CURLOPT_FAILONERROR, 1);

    $x = curl_exec($resURL);

    $intReturnCode = curl_getinfo($resURL, CURLINFO_HTTP_CODE);
    curl_close ($resURL);

    if ($intReturnCode != 200 && $intReturnCode != 302 && $intReturnCode != 304) {
       return false;
    }else{
        return true ;
    }
} 
function readHeader($data)
{
return $data;
}
?> 
</body>

d.shankar · October 6, 2007

You still didnt add the timeout ... lol

MadTechie · October 6, 2007

it should display them as they are done, (hence the flush()

try changing

flush();

to

ob_flush();
flush();

as for the timeout

it depends on whats the slow part,

comment out the following code

	$data = file_get_contents($URL);
	unset($data);//dump

see if its quicker

as for the curl

try adding this

curl_setopt($resURL, CURLOPT_TIMEOUT, 1);

Works for me

<head>Test Cheker 1.0v</head>
<body>
<?php
$URLarray = array(
"http://www.phpfreaks.com/"	);


echo "<table>\n";
echo "  <tr>\n";
echo "      <td>\n";
echo "URL"; 
echo "      </td>\n";
echo "      <td>\n";
echo "Time"; 
echo "      </td>\n";
echo "  </tr>\n";

foreach($URLarray as $URL)
{
echo "  <tr>\n";
$mtime = microtime(); 
$mtime = explode(' ', $mtime); 
$mtime = $mtime[1] + $mtime[0]; 
$starttime = $mtime; 
if( url_exists($URL) )
{
	$data = file_get_contents($URL);
	unset($data);//dump	
	$mtime = microtime(); 
	$mtime = explode(" ", $mtime); 
	$mtime = $mtime[1] + $mtime[0]; 
	$endtime = $mtime; 
	$totaltime = ($endtime - $starttime); 
	$UrlResults[$URL] = array("Access" => true, "Time"=>$totaltime); 
	echo "      <td>\n";
	echo "$URL"; 
	echo "      </td>\n";
	echo "      <td>\n";
	echo "$totaltime seconds."; 
	echo "      </td>\n";
}else{
	$UrlResults[$URL] = array("Access" => false); 
	echo "      <td>\n";
	echo "$URL"; 
	echo "      </td>\n";
	echo "      <td>\n";
	echo "Failed"; 
	echo "      </td>\n";
}
echo "  </tr>\n";
flush();
}
echo "</table>\n";

//Results from loop (if you want)
//foreach($UrlResults as $URLs => $Tests)
//{
//	echo "$URLs: ";
//	if($Tests['Access'])
//	{
//		echo "Works access time:".$Tests['Time'];
//	}else{
//		echo "Failed";
//	}
//	echo "<br />";
//}


function url_exists($strURL)
{
    $resURL = curl_init();
    curl_setopt($resURL, CURLOPT_URL, $strURL);
curl_setopt($resURL, CURLOPT_BINARYTRANSFER, 1);
    curl_setopt($resURL, CURLOPT_HEADERFUNCTION, 'readHeader');
    curl_setopt($resURL, CURLOPT_FAILONERROR, 1);
curl_setopt($resURL, CURLOPT_TIMEOUT, 1); 

    $x = curl_exec($resURL);

    $intReturnCode = curl_getinfo($resURL, CURLINFO_HTTP_CODE);
    curl_close ($resURL);

    if ($intReturnCode != 200 && $intReturnCode != 302 && $intReturnCode != 304) {
       return false;
    }else{
        return true ;
    }
} 
function readHeader($data)
{
return $data;
}
?>

</body>

jjmusicpro · October 6, 2007

I only put to check phpfreaks.com for now...

let me run this and see what happens....

thanks again~!

MadTechie · October 6, 2007

Test Cheker 1.0v
URL Time

http://www.phpfreaks.com/ 1.35130906105 seconds.

jjmusicpro · October 6, 2007

Whats the command again to check if i have curl running?

d.shankar · October 6, 2007

<?php
phpinfo();
?>

then ctrl+F and search for curl

MadTechie · October 6, 2007

phpinfo();

to check its installed

curl
CURL support enabled

jjmusicpro · October 6, 2007

perfect, looks like it works...

now how to i pull those values out of the db? lol

jjmusicpro · October 6, 2007

Is there a way to list the URL's first, then as i go through them, put "OK" if they load within 3 seconds, and then display time?

If i have like 30 urls, the page takes forever to load, i was thinking show the URLS as they are getting done ??

jjmusicpro · October 6, 2007

i try to run the script with multiple sites to check but it just comes up blank...

http://neckter.com/linkchecker.php

<head><title>Test Cheker 1.0v</title></head>
<body>
<b>Test Urls 1.0v</b><p>
<?php
$URLarray = array(
"http://www.phpfreaks.com/"	);
"http://www.ebay.com/"	);
"http://www.myspace.com/"	);


echo "<table>\n";
echo "  <tr>\n";
echo "      <td>\n";
echo "URL"; 
echo "      </td>\n";
echo "      <td>\n";
echo "Time"; 
echo "      </td>\n";
echo "  </tr>\n";

foreach($URLarray as $URL)
{
echo "  <tr>\n";
$mtime = microtime(); 
$mtime = explode(' ', $mtime); 
$mtime = $mtime[1] + $mtime[0]; 
$starttime = $mtime; 
if( url_exists($URL) )
{
	$data = file_get_contents($URL);
	unset($data);//dump	
	$mtime = microtime(); 
	$mtime = explode(" ", $mtime); 
	$mtime = $mtime[1] + $mtime[0]; 
	$endtime = $mtime; 
	$totaltime = ($endtime - $starttime); 
	$UrlResults[$URL] = array("Access" => true, "Time"=>$totaltime); 
	echo "      <td>\n";
	echo "$URL"; 
	echo "      </td>\n";
	echo "      <td>\n";
	echo "$totaltime seconds."; 
	echo "      </td>\n";
}else{
	$UrlResults[$URL] = array("Access" => false); 
	echo "      <td>\n";
	echo "$URL"; 
	echo "      </td>\n";
	echo "      <td>\n";
	echo "Failed"; 
	echo "      </td>\n";
}
echo "  </tr>\n";
flush();
}
echo "</table>\n";

//Results from loop (if you want)
//foreach($UrlResults as $URLs => $Tests)
//{
//	echo "$URLs: ";
//	if($Tests['Access'])
//	{
//		echo "Works access time:".$Tests['Time'];
//	}else{
//		echo "Failed";
//	}
//	echo "<br />";
//}


function url_exists($strURL)
{
    $resURL = curl_init();
    curl_setopt($resURL, CURLOPT_URL, $strURL);
curl_setopt($resURL, CURLOPT_BINARYTRANSFER, 1);
    curl_setopt($resURL, CURLOPT_HEADERFUNCTION, 'readHeader');
    curl_setopt($resURL, CURLOPT_FAILONERROR, 1);
curl_setopt($resURL, CURLOPT_TIMEOUT, 1); 

    $x = curl_exec($resURL);

    $intReturnCode = curl_getinfo($resURL, CURLINFO_HTTP_CODE);
    curl_close ($resURL);

    if ($intReturnCode != 200 && $intReturnCode != 302 && $intReturnCode != 304) {
       return false;
    }else{
        return true ;
    }
} 
function readHeader($data)
{
return $data;
}
?>

</body>

MadTechie · October 6, 2007

OK done

please note the multiple sites (no comma on the last one)

<head>Test Cheker 1.0v</head>
<script language="javascript">
function display(id ,str)
{
document.getElementById(id).innerHTML = str;
}
</script>
<body>
<?php
$URLarray = array(
"http://www.phpfreaks.com/",
"http://www.ebay.com/",
"http://www.myspace.com/"
	);


echo "<table>\n";
echo "  <tr>\n";
echo "      <td>\n";
echo "URL"; 
echo "      </td>\n";
echo "      <td>\n";
echo "Time"; 
echo "      </td>\n";
echo "  </tr>\n";

//URLS
foreach($URLarray as $K => $URL)
{
echo "  <tr>\n";

echo "      <td>\n";
echo "			$URL"; 
echo "      </td>\n";
echo "      <td>\n";
echo "			<div id='$K'>please Wait</div>"; 
echo "      </td>\n";

echo "  </tr>\n";
ob_flush();
flush();
}
echo "</table>\n";


//Times
foreach($URLarray as $K => $URL)
{
$mtime = microtime(); 
$mtime = explode(' ', $mtime); 
$mtime = $mtime[1] + $mtime[0]; 
$starttime = $mtime; 
if( url_exists($URL) )
{
	$data = file_get_contents($URL);
	unset($data);//dump	
	$mtime = microtime(); 
	$mtime = explode(" ", $mtime); 
	$mtime = $mtime[1] + $mtime[0]; 
	$endtime = $mtime; 
	$totaltime = ($endtime - $starttime); 
	$UrlResults[$URL] = array("Access" => true, "Time"=>$totaltime); 
	echo "			<script language='javascript'>display('$K', '$totaltime seconds');</script>"; 
}else{
	$UrlResults[$URL] = array("Access" => false); 
	echo "			<script language='javascript'>display('$K', 'Failed');</script>"; 
}
ob_flush();
flush();
}


function url_exists($strURL)
{
    $resURL = curl_init();
    curl_setopt($resURL, CURLOPT_URL, $strURL);
curl_setopt($resURL, CURLOPT_BINARYTRANSFER, 1);
    curl_setopt($resURL, CURLOPT_HEADERFUNCTION, 'readHeader');
    curl_setopt($resURL, CURLOPT_FAILONERROR, 1);
curl_setopt($resURL, CURLOPT_TIMEOUT, 1); 

    $x = curl_exec($resURL);

    $intReturnCode = curl_getinfo($resURL, CURLINFO_HTTP_CODE);
    curl_close ($resURL);

    if ($intReturnCode != 200 && $intReturnCode != 302 && $intReturnCode != 304) {
       return false;
    }else{
        return true ;
    }
} 
function readHeader($data)
{
return $data;
}
?>

</body>

Sign In

Link Checker + Time it took

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Archived

Important Information