File() Function Failing To Open/Load httpS Pages ?

[phpsane]

Hi,

 

It seems file() is having problems opening httpS pages. Can anyone test this on your end and confirm ?

As you can see, I fed a valid url. I actually grabbed the following code from that particular url.

 

Warning: file(https://oscarliang.com/six-ways-retrieving-webpage-content-php/): failed to open stream: HTTP request failed! HTTP/1.1 403 Forbidden in  C:\xampp\htdocs\php\filter.php on line 15

Warning: implode(): Invalid arguments passed in C:\xampp\htdocs\php\filter.php on line 17

<?php
 
/*
ERROR HANDLING
*/
declare(strict_types=1);
ini_set('display_errors', '1');
ini_set('display_startup_errors', '1');
error_reporting(E_ALL);
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
 
 
$url='https://oscarliang.com/six-ways-retrieving-webpage-content-php/';
// using file() function to get content
$lines_array=file($url);
// turn array into one variable
$lines_string=implode('',$lines_array);
//output, you can also save it locally on the server
echo $lines_string;
 
 
 
?>

As you can see, I fed a valid url. I actually grabbed the code from that url.

Edited September 3, 2017 by phpsane

[phpsane]

Fed the same url to the function file_get_contents() but still no luck. I get error:

 

Warning: file_get_contents(https://oscarliang.com/six-ways-retrieving-webpage-content-php/): failed to open stream: HTTP request failed! HTTP/1.1 403 Forbidden in  C:\xampp\htdocs\php\filter.php on line 16

Fatal error: Uncaught TypeError: htmlspecialchars() expects parameter 1 to be string, boolean given in C:\xampp\htdocs\php\filter.php:18 Stack trace: #0 C:\xampp\htdocs\php\filter.php(18): htmlspecialchars(false) #1 {main} thrown in C:\xampp\htdocs\php\filter.php on line 18

 

<?php
 
/*
ERROR HANDLING
*/
declare(strict_types=1);
ini_set('display_errors', '1');
ini_set('display_startup_errors', '1');
error_reporting(E_ALL);
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
 
 
$url='https://oscarliang.com/six-ways-retrieving-webpage-content-php/';
 
//file_get_contents() reads remote webpage content
$lines_string=file_get_contents($url);
//output, you can also save it locally on the server
echo htmlspecialchars($lines_string);
?>

[phpsane]

No luck with fopen() either.

 

Warning: fopen(https://oscarliang.com/six-ways-retrieving-webpage-content-php/): failed to open stream: HTTP request failed! HTTP/1.1 403 Forbidden in  C:\xampp\htdocs\php\filter.php on line 16

Fatal error: Uncaught TypeError: fread() expects parameter 1 to be resource, boolean given in C:\xampp\htdocs\php\filter.php:21 Stack trace: #0 C:\xampp\htdocs\php\filter.php(21): fread(false, 1024) #1 {main} thrown in C:\xampp\htdocs\php\filter.php on line 21

 

 

 

<?php
 
/*
ERROR HANDLING
*/
declare(strict_types=1);
ini_set('display_errors', '1');
ini_set('display_startup_errors', '1');
error_reporting(E_ALL);
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
 
 
$url='https://oscarliang.com/six-ways-retrieving-webpage-content-php/';
 
//fopen opens webpage in Binary
$handle=fopen($url,"rb");
// initialize
$lines_string="";
// read content line by line
do{
$data=fread($handle,1024);
if(strlen($data)==0) {
break;
}
$lines_string.=$data;
}while(true);
//close handle to release resources
fclose($handle);
//output, you can also save it locally on the server
echo $lines_string;
 
?>

[archive]

What do you think

 

Forbidden

means?

[phpsane]

What do you think

means?

 

It means I am doing something I should not do.

As far as I know, fopen() or file() or fread() should be used to open a file on the server that the php script is hosted on and I should not feed a url. Whereas these 2 codes feed urls to these functions. I'm guessing, that is the whole problem. Do you agree to this ?

In other words, the codes I grabbed from that url is full of flaws. Internet full of buggy php codes!

 

https://oscarliang.com/six-ways-retrieving-webpage-content-php/

Edited September 4, 2017 by phpsane

[kicken]

fopen/file can open URL's (if allow_url_fopen is enabled). Your problem is the specific URL you are trying to open is denying your script.

 

Out of curiosity, I spent a few minutes trying to figure out why. It seems that server denys access if no User-agent: header is specified.

 

I'll leave resolving that problem as an exercise for you to do.

Edited September 4, 2017 by kicken

[phpsane]

fopen/file can open URL's (if allow_url_fopen is enabled). Your problem is the specific URL you are trying to open is denying your script.

 

Out of curiosity, I spent a few minutes trying to figure out why. It seems that server denys access if no User-agent: header is specified.

 

I'll leave resolving that problem as an exercise for you to do.

 

Mmm. I'm guessing if there is no UserAgent then the website jumps to conclusion that I am a bot and starts denying the page.

Possibility correct, you assume ?