How do I make fopen() work with a http url

[designergav]

Hi,

I've got this script which works if the file I'm reading is in the same directory as the php file but I need to get it to read a remote file. I get this error message

CGI Error
The specified CGI application misbehaved by not returning a complete set of HTTP headers.

 

Here's the script.

 

<?PHP include ("dbconnectionfile.php"); 

$filename = "http://www.mysite.co.uk/myfiles/file_to_read.csv";
$fd = fopen ($filename, "r");
$contents = fread ($fd,filesize ($filename));
fclose ($fd);

$delimiter = ",";
$splitcontents = explode($delimiter, $contents);
$counter = "";
//print $contents;

//remove any single quotes
$noquotes = str_replace("'", "", $contents);
//remove linebreaks and replace with ),(
$linebreak = str_replace("\r", "),(", $noquotes);
//replace " with '
$singlequotes = str_replace("\"", "'", $linebreak);
$singlequotes = $singlequotes."'end','end','end";
//remove everything before 1st currency
$trimmed = strstr($singlequotes, "AED");
//print "<H1>trimmed</h1><p> $trimmed";

//DELETE EXISTING DATA
$sql = "TRUNCATE TABLE mytable";
$result = mysql_query($sql)
or die(mysql_error());

//INSERT currencies into DB
$sql = "INSERT into mytable (csymbol, cname, crate) VALUES ('$trimmed"."')";
$result = mysql_query($sql)
or die(mysql_error());
if (!$result) {
print "<p><h3>Error inserting data</h3>";
} else {
print "<p><h3>CURRENCY UPDATE DONE!!</h3>";
} 

?>

 

I'm still fairly new to php/mysql and it's taken 3months (off and on) to get this far and i'm stumbling at this final hurdle.

 

I've looked at loads of forums google searches and everyone uses fopen("http...") with no mention of errors. Any ideas

For testing I'm using my own site as the url and I know the script is working and it finds the file without the http bit.

 

Any ideas?

DesignerGav

 

[Balmung-San]

Do you have allow_url_fopen set to true in your php.ini?

[designergav]

I'm not sure. Our host doesn't allow access to top level htaccess etc files. If the name is relative eg ("myfile.csv") does that mean it's not a url in terms of the function required?

 

It works fine for relative filenames.

 

If it's not a url then I guess I need to speak to our host really nicely and ask them to set url_fopen. Although I won't hold my breath.

[Balmung-San]

You can easily find out if it's open or not. Create a file named whatever you want (I suggest phpinfo.php), with the following contents:

 

<?php
phpinfo();
?>

 

That will print out your PHP configuration. Under Configuration -> PHP Core it should be the second one down.

 

Also, php.ini is the config file for PHP, where as .htaccess files are for apache.

[designergav]

Thanks that's a really useful info page.

 

allow_url_fopen is set to ON for local and master.

[Balmung-San]

Yes, it's a very good page for quickly looking at your config data.

 

Make sure the http stream is registered (which it should be, but always worth a check). It's on the same info page, in the first big block. There should be a spot for Registered PHP Streams.

[designergav]

It says:

Registered php streams: php, http, ftp, compress.zlib

[Balmung-San]

I've looked around the php bug listing and found a possible solution. Add this in your code before sending your fopen.

 

ini_set('user_agent', 'PHP/4.3.2');

 

Unless you're using a different version of PHP. In which case it'll be PHP/your version number.

[designergav]

I'm running 4.4.2 on Windows NT ISVWEB 5.2 build 3790. I'm afraid I still get the error message.

 

Do I need to refer to the HTTP_USER_AGENT which currently says Mozilla/4.0 (compatible; MSIE 5.17; Mac_PowerPC)

Or is that just info about my machine.

 

Thanks for all this by the way.

[Balmung-San]

That's just info about your machine. The HTTP_USER_AGENT is in reference to the user actually using the page. Very useful for optimizing your layout for different browsers.

 

You have a very weird build for your web server, and I haven't been able to find anything relevant in the PHP bug database.

 

One thing you may want to check on, does the user that runs the webserver have permissions to open the file?

[designergav]

Yes I know it's wierd. It's because the client has an exchange server handling incoming emails. Microsoft eh? The world would be better off… Well we all know what.

 

I assume you mean someone at the hosting company having access. Which file do you mean? The csv file I'm trying to read? I'll have to check but I that they can see open the php files as they've checked something for me in the past.

 

One other thing I've spotted is the upload_tmp_dir. Could I store a file there are refer to it. I could in theory write a script to download my csv file (from the http site not my local site) and save it to this directory and then refer to that location for the  fopen script.

[Balmung-San]

That might be your best option. I know Windows has always been picky with file permissions and such, and NT is no exception. Not to mention I've never even heard of ISVWEB.

 

What I was asking about is if the user that the web server runs under (for example, on Linux machines apache runs as the user nobody, and IIS usually runs under IUSR_machinename) has access to read the file.

 

Also, does the machine run an ftp server as well? You might want to try using ftp protocol instead of http if possible.

[designergav]

ISVWEB appears under the name _ENV["COMPUTERNAME"]. Server software is Microsoft IIS/6.0.

 

Yes there is an ftp server for the website but I believe it runs on a seperate server. The file I need to read is a licensed file that is accessed via a HTTP GET request and unfortunately I have no control over that so can't use ftp.

 

It's starting to look like I might need to rethink things. I have another website www.mysite2.co.uk that lives on a linux server and is just a host for the mysql db for www.mysite.co.uk I could try moving the files there and seeing what happens.

 

It may solve the problem and as the db lives there might make it easier anyway. I'll try that and see what happens.

 

Thanks for your help and I'll let you know if it works.

Gavin

[Balmung-San]

Alright, hope that works for you.

[designergav]

Well. PLAN C anyone.

 

I now get these messages:

Warning: filesize(): Stat failed for http://www.mysite2.co.uk/myfiles/file_to_read.csv (errno=2 - No such file or directory) in /home/fhlinux185/a/mysite2.co.uk/user/htdocs/myfiles/test.php on line 11

Warning: fread(): Length parameter must be greater than 0. in /home/fhlinux185/a/mysite2.co.uk/user/htdocs/myfiles/test.php on line 11
Column count doesn't match value count at row 1

Doesn't seem to find the file to read. Column count error is because the contents of the file form part of the INSERT.

 

the files are there because I can type the direct url to both the test and csv files and see them.

[designergav]

mysite2.co.uk is hosted on a stricter webhosting companies server so I imagine many of the settings required are turned off for security and I know from past dealings that I can't change anything on their server.

 

I think this may be a dead end.

Oh well.

[Balmung-San]

So what is it that this file was supposed to do? Perhaps there's some kind of alternative solution.

[designergav]

I will be provided a csv file daily which has 200 currency exchange rates. I need to get them from that csv file into a mysql table to pull into a website as part of dynamic content. This needs to update the rates every day

 

The file is licensed and I can only request it once a day and I cannot do anything other than read/save the file. Obviously I'd like to automate the process so that windows scheduler or cronjob can run the php script automatically.

 

 

[designergav]

Hold the phone. UPDATE:

 

If i use an asolute link to the file based on the path shown in my error message: /home/fhlinux185/a/mysite2.co.uk/user/htdocs/myfiles/file_to_read.csv

 

it works. Now all I have to do is find out what the equivalent path would be for the actual location of the real file.

 

Does this look like it would would in real life?

 

NO GUESS NOT. If I move the php file but still refer to that location for the file to read it doesn't work. I suppose because the path is only relative to the php file not an absolute path to the file on the server

[Balmung-San]

Don't we all wish we didn't live in real life.

 

It seems pretty okay, though that's a physical path based on the webserver, so you can't just put that into the other server's script.

[designergav]

Well. I'm officially baffled. How can it find /myfile.csv but not http://www.mysite.co.uk/files/myfile.csv

 

I'm all out of ideas. I though about saving the file to my site but to access the file and save surely I'll have the same problem with http as don't i need to "read" it to save it?

 

Could cgi scripts help here and how would I go about using them?

[shoz]

Well. I'm officially baffled. How can it find /myfile.csv but not http://www.mysite.co.uk/files/myfile.csv

 

I'm all out of ideas. I though about saving the file to my site but to access the file and save surely I'll have the same problem with http as don't i need to "read" it to save it?

 

Could cgi scripts help here and how would I go about using them?

 

If you're trying to access a file that is hosted on another site you may also want to try "file_get_contents()"

 

$string = file_get_contents('http://www.mysite.co.uk/files/myfile.csv');
var_dump($string);

 

 

 

[designergav]

I get the same GCI error as with fopen. Well I'm off to today. I'll have to pick this up at a later date.

 

Thanks for everyone's help today especially you Balmung-San.

 

No doubt I'll be back in this thread soon. If anyone finds the answer before me. Let me know.

 

Gav

[Balmung-San]

Thanks for everyone's help today especially you Balmung-San.

 

I was at work, what else would I have been doing? xD