get file contents - time out

[imperium2335]

Is there are way to make the get_file_contents function time out if the target site is not responding, and then move onto the next?

 

At the moment my script just times out because of the typical php timout.

 

Thanks in advance.

[Daniel0]

Yeah, just pass it a context resource specifying the timeout value (it's in the manual).

[imperium2335]

ive tried that but its not working, just behaving as normal

$ctx = stream_context_create(array('http' => array('timeout' <= 1)));

 

$var= @file_get_contents($thisone, 0, $ctx) ;

[Daniel0]

That's because you defined your array incorrectly.

 

<= is a comparison operator, so what you will have is a in the 'http' index is an array containing the boolean value of 'timeout' <lower than or equal to> 1. What you're looking for is:

$ctx = stream_context_create(array('http' => array('timeout' => 1)));

[imperium2335]

that does not work either

[Daniel0]

"Doesn't work" doesn't help me help you.

 

It "works" for me.

 

I have test2.php looking like this:

<?php
sleep(10);
echo 'hi';

 

And another file looking like this:

<?php
$ctx = stream_context_create(array('http' => array('timeout' => 1)));

$var= file_get_contents('http://localhost/test2.php', 0, $ctx);

echo $var;

 

Which results in

Warning: file_get_contents(http://localhost/test2.php) [function.file-get-contents]: failed to open stream: HTTP request failed! in C:\xampp\htdocs\test.php on line 4

 

As you would expect.

 

If I set timeout to a value higher than 10 I'll get the 'hi' output.

[imperium2335]

so timeout is just a variable i need to declare? at the moment i dont have that.

[imperium2335]

what i want is the function to time out if the url is taking to long to respond, so the loop can continue and move onto the next url (it is looping through a list of urls and retrieving the content).

[Daniel0]

Right, and that is possible. The timeout is a float. If the time spent on retrieving the resource is larger than that float value it will fail with a Warning as demonstrated in the example.

 

Just set the timeout to the maximum acceptable time for you.

[imperium2335]

but wont the sleep() function cause the whole script to pause?

[Daniel0]

The sleep() I added in another file was just to make sure it took long enough for the file retrieval to timeout. It was to emulate high network latency.

[imperium2335]

sorry, I still dont understand Where is the length of timeout coming from?

 

$ctx = stream_context_create(array('http' => array('timeout' => 1)));

 

$currentblog = @file_get_contents($thisone, 0, $ctx) ;

 

or sleep(10)?

 

Have tried everything you have said so far but it doesn't work

[Daniel0]

$ctx = stream_context_create(array('http' => array('timeout' => 1)));

 

You set it there, to whatever value you want. Ignore sleep(), that was for my testing purposes in a different script.

[imperium2335]

i thought u said that => 1 was boolean for true? as in 0 is false? it is still not working though thanks for trying so far.

[Daniel0]

No, you did <=, which is a binary operator that returns a boolean value. I did =>, which is quite different.

 

You might want to read the Language Reference chapter in the manual.

 

Again, saying "it doesn't work" really isn't of much help to me. Doesn't work can mean hundreds of different things. You need to tell me how it doesn't work, i.e., what did you expect and what happened instead?

[MadTechie]

Yes it is.. works for me

 

Here's an idea maybe, if you give a little bit of useful information or here's an idea maybe post what you have and a description of what you want it to do and what it is or is not doing, you know considing we are spending time to help maybe a little bit of your time would be a valid investment

[imperium2335]

it is going through a list of addresses and assigning the output to a variable which is then analysed. It then repeats for the next address in the list. But if an address is taking forever to respond the script will time out (as expected) so it doesn't scan the rest of the addresses in the list after the one that takes forever and never loads.

[Daniel0]

Right, something like

<?php
$addresses = array(
'http://foo.com/something.php',
'http://example.com/helloWorld.py',
);

$context = stream_context_create(array('http' => array('timeout' => 1)));

$values = array();
$oldErrorReporting = error_reporting(0); // faster than using @ on each iteration
foreach ($addresses as $address) {
$var = file_get_contents($address, $context);
}
error_reporting($oldErrorReporting);

 

If something takes over 1 second it will be skipped as per the timeout value.

[MadTechie]

Maybe worth noting

 

Whats the timeout for the script.. if each site took 1 second and you had 60 sites, but your timeout was 30 then the script will timeout, with that inmind it may be worth adding

set_time_limit(0);

 

Also

   $var = file_get_contents($address, $context);

should probably be something like

   $var = file_get_contents($address, $context);
   $values[] = ($var===false)?false:true;

 

 

[imperium2335]

Hi Mad, once a sites contents are retrieved successfully, 30 seconds is added onto the time out.

[MadTechie]

If you say so.. theirs no way i could know thast with seeing the code!

[imperium2335]

$Target_Fragment = $_POST['frag'] ;
if($Target_Fragment == NULL) header("Location: default.html") ;
$Scan_Time_Limit = $_POST['tout'] ; // The maximum time allowed to scan one site.
echo "<head><style type=\"text/css\">
#probar {width:136px;height:16px;background:url('probar.gif') bottom no-repeat ;*background:url('probar.gif') top no-repeat ;}
</style></head>" ;
$ext = explode(".", ($_FILES['upload']['name'])) ;

$ext = $ext[1] ;

if($ext != "txt") {
echo "File must be a TXT file, <a href=\"default.html\">go back</a> and try again." ;
exit() ;
}
else $listfile = ($_FILES['upload']['tmp_name']);

$ctx = stream_context_create(array('http' => array('timeout' => 3))); 

$pb = 0 ;

$stuff = file_get_contents($listfile) ;

$urls = explode("http://", $stuff) ;

$stuffcount = count($urls)-1 ;

$found = 0 ;

$notfound = 0 ;

$scanned = 0 ;

$successes = array() ;

$fails = array() ;

array_shift($urls) ;

$a=0;$b=0;$c=0;$d=0;$e=0;$f=0;$g=0;$h=0;$ip=0;$j=0;$int=0; //Progress bar stops.
echo "Scanning $stuffcount  from the uploaded text file " . ($_FILES['upload']['name']) . ".<br/>" ;
echo "Standby" ;

for($i = 0; $i < $stuffcount; $i++)	{
$thisone = "http://" . $urls[$i] ;
    
    
    set_time_limit($Scan_Time_Limit) ;
$currentitem = @file_get_contents($thisone, $ctx) ;

$bool = stristr($currentitem, $Target_Fragment) ;

if($bool == TRUE) {
	$found++ ;
	array_push($successes, $urls[$i]) ;
}
else {
	$notfound++ ;
	array_push($fails, $urls[$i]) ;
}
$scanned++ ;
    
  
    

$pcomplete = ceil((($scanned / $stuffcount) * 100)) ;

if($int == 0) echo "<div id=\"probar\">" ;
$int = 1 ;
if($pcomplete > 0 & $a == 0){echo "<img src=\"pip.jpg\"> " ; $a = 1;}
if($pcomplete > 20 & $b == 0){echo "<img src=\"pip.jpg\"> " ; $b = 1;}
if($pcomplete > 30 & $c == 0){echo "<img src=\"pip.jpg\"> " ; $c = 1;}
if($pcomplete > 40 & $d == 0){echo "<img src=\"pip.jpg\"> " ; $d = 1;}
if($pcomplete > 50 & $e == 0){echo "<img src=\"pip.jpg\"> " ; $e = 1;}
if($pcomplete > 60 & $f == 0){echo "<img src=\"pip.jpg\"> " ; $f = 1;}
if($pcomplete > 70 & $g == 0){echo "<img src=\"pip.jpg\"> " ; $g = 1;}
if($pcomplete > 80 & $h == 0){echo "<img src=\"pip.jpg\"> " ; $h = 1;}
if($pcomplete > 90 & $ip == 0){echo "<img src=\"pip.jpg\"> " ; $ip = 1;}
if($pcomplete > 95 & $j == 0){echo "<img src=\"pip.jpg\"></div>" ; $j = 1;}


}

echo "<p>Scan complete</p>" . "<p>$scanned  were checked for the code fragment: $Target_Fragment </p>" . "<p>$found  contain the fragment.</p><p>$notfound  do not contain the fragment.</p>" ;

echo "
<table width=\"1000\" border=\"1\">
<tr><td><strong>URL</strong></td><td><strong>Found?</strong></td>
" ;

$i = 0 ;
$x = 0 ;

for($z = 0; $z < $scanned; $z++) {

echo "<tr><td width=\"500\"><a href=\"http://$urls[$z]\" target=\"_blank\">$urls[$z]</a>" ;

if(in_array($urls[$z], $successes)) echo "<td><strong>Found!</strong></td></tr>" ;
else echo "<td width=\"500\"></td></tr>" ;
}
echo "</table>" ;

[imperium2335]

Any idea why the timeout still wont work?

[Daniel0]

Seriously, what do you mean with "it doesn't work"? It can mean hundreds of things.