fcurl

[sanchez77]

Hi Everyone,

 

So a while ago I wrote this code to read a text file from a URL, process the data and put into a table then display the data. It worked great until my ISP stopped me from using file(), so I guess I need to use curl to open the file instead of file.

 

I have been trying with no success and I am looking for help. Here is the original code

 

<?php


  include "connect.php";
if (!$con)
  {
  die('Could not connect: ' . mysql_error());
  }

$query = "DELETE FROM buoy44025";
$result = mysql_query($query);


$wx = array_map('trim',file("http://www.ndbc.noaa.gov/data/realtime2/44025.txt"));
$newwx = array();
foreach($wx as $i => $line) {
   if ($i > 1 && $i <  {
        $tmp = array_filter(explode(' ',$line));
        $q = "insert into buoy44025 (year,month,day,hour,min,wdir,wspd,gst,waveht,dpd,apd,mwd,pres,atmp,wtmp,dewp,vis,ptdy,tide) values ('" . implode("','",$tmp) . "')";
        $rw = mysql_query($q) or die("Problem with the query: $q<br>" . mysql_error());
   }
}

echo "</table>";mysql_close($con);

?>

 

Does anyone know how i could implement the curl() function instead of the file() function and keep the page working?

 

Any help is greatly appreciated. Cheers

[premiso]

function getFile($url) {
    $ch = curl_init();

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

    $file = curl_exec($ch);
    curl_close($ch);

    return $file;
}

// Then replace the file call with getFile
getFile("http://www.ndbc.noaa.gov/data/realtime2/44025.txt");

 

Reading the manual is often the best bet. I took this straight from the curl_init page, just added the return transfer and wrapped it in a function for you.

[sanchez77]

Thanks, I read the manual, couldn't get it to work, hence why I am here. I tried inputting your code but it returns a blank page. Is this the correct implementation?

 

<?

function getFile($url) {    
$ch = curl_init();  
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);  
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);  
$file = curl_exec($ch); 
curl_close($ch);    
return $file;
}

  include "connect.php";
if (!$con)
  {
  die('Could not connect: ' . mysql_error());
  }

$query = "DELETE FROM buoy44025";
$result = mysql_query($query);


$wx = array_map(getFile("http://www.ndbc.noaa.gov/data/realtime2/44025.txt");

$newwx = array();
foreach($wx as $i => $line) {
   if ($i > 1 && $i <  {
        $tmp = array_filter(explode(' ',$line));
        $q = "insert into buoy44025 (year,month,day,hour,min,wdir,wspd,gst,waveht,dpd,apd,mwd,pres,atmp,wtmp,dewp,vis,ptdy,tide) values ('" . implode("','",$tmp) . "')";
        $rw = mysql_query($q) or die("Problem with the query: $q<br>" . mysql_error());
   }
}
?>

[premiso]

Should be, you may have to try adding the header and setting the USERAGENT via the curl_setopts to trick the page into thinking it is a browser requesting it.

 

function getFile($url) {
    $ch = curl_init();

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HEADER, 1);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

    $file = curl_exec($ch);
    curl_close($ch);

    return $file;
}

 

See what that does.

[sanchez77]

That worked for returning the page. But I have an error now that says 'Warning: array_map() [function.array-map]: Argument #2 should be an array '

 

Since it has changed to the getfile(), do i need to do something to make it an array?

 

<?
error_reporting(E_ALL);
ini_set("display_errors", 1);

function getFile($url) { 
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
$file = curl_exec($ch);
curl_close($ch);
return $file;
}

include "connect.php";
if (!$con)
  {
  die('Could not connect: ' . mysql_error());
  }

$query = "DELETE FROM buoy44025";
$result = mysql_query($query);

$wx = array_map('trim',getfile("http://www.ndbc.noaa.gov/data/realtime2/44025.txt"));
$newwx = array();
foreach($wx as $i => $line) {
   if ($i > 1 && $i <  {
        $tmp = array_filter(explode(' ',$line));
        $q = "insert into buoy44025 (year,month,day,hour,min,wdir,wspd,gst,waveht,dpd,apd,mwd,pres,atmp,wtmp,dewp,vis,ptdy,tide) values ('" . implode("','",$tmp) . "')";
        $rw = mysql_query($q) or die("Problem with the query: $q<br>" . mysql_error());
   }
}
?>

[premiso]

explode. file reads in a file and splits it at the line character.

 

function getFile($url) {
    $ch = curl_init();

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HEADER, 1);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

    $file = curl_exec($ch);
    curl_close($ch);

    return explode("\n", $file);
}

 

Should return an array.

[sanchez77]

Thanks, I apprecaite it. Were almost there. So I am getting this error:

 

Problem with the query: insert into buoy44025 (year,month,day,hour,min,wdir,wspd,gst,waveht,dpd,apd,mwd,pres,atmp,wtmp,dewp,vis,ptdy,tide) values ('Server:','Apache')

Column count doesn't match value count at row 1

 

I'm not sure why it has values Server, Apache cosidering the first few rows look like this:

 

#YY  MM DD hh mm WDIR WSPD GST  WVHT  DPD  APD MWD  PRES  ATMP  WTMP  DEWP  VIS PTDY  TIDE

#yr  mo dy hr mn degT m/s  m/s    m  sec  sec degT  hPa  degC  degC  degC  nmi  hPa    ft

2011 07 27 17 50 290  6.0  7.0  0.9    6  4.5 138 1012.0  23.9  23.6  16.2  MM +0.0    MM

2011 07 27 16 50 290  5.0  6.0  0.9    7  4.4 149 1012.0  23.5  23.5  17.0  MM +0.0    MM

2011 07 27 15 50 310  5.0  6.0  0.9    7  4.4 152 1011.9  23.3  23.4  13.7  MM +0.0    MM

 

 

Do you know when it has Server, Apache for values? Or were it got those values from?

 

Thanks for your help.

[premiso]

Dump the $file in the function and see what is coming. It could be a server error, that would be my bet. var_dump would do the trick to see the data you are getting.

[sanchez77]

I put this in

echo "--";
var_dump($file);
echo "--";

 

And it says Null, that can't be right? Can it?

[premiso]

I just took the curl code and ran it on my server.

 

  string(93) "2011 06 13 16 50 330  1.0  2.0   0.8     7   4.8 105 1012.4  18.1  19.3   9.7   MM +0.4    MM"
  [1075]=>
  string(93) "2011 06 13 15 50  20  2.0  3.0   0.9     6   4.8 107 1012.5  17.5  18.8   9.3   MM +0.7    MM"
  [1076]=>
  string(93) "2011 06 13 14 50  30  4.0  5.0   0.8     7   4.5 101 1012.4  17.5  18.6  11.0   MM +1.2    MM"

 

Seems to be working just fine on my end. Not sure where your hold up would be. Did it ever work?

[sanchez77]

I'm glad it's working, sort of. This is the error i get:

 

ArrayProblem with the query: insert into buoy44025 (year,month,day,hour,min,wdir,wspd,gst,waveht,dpd,apd,mwd,pres,atmp,wtmp,dewp,vis,ptdy,tide) values ('Server:','Apache')

Column count doesn't match value count at row 1

 

So I guess I have a problem with this code

 

$wx = array_map('trim',getFile("http://www.ndbc.noaa.gov/data/realtime2/44025.txt"));

$newwx = array();

foreach($wx as $i => $line) {
   if ($i > 1 && $i <  {
        $tmp = array_filter(explode(' ',$line));
        $q = "insert into buoy44025 (year,month,day,hour,min,wdir,wspd,gst,waveht,dpd,apd,mwd,pres,atmp,wtmp,dewp,vis,ptdy,tide) values ('" . implode("','",$tmp) . "')";
        $rw = mysql_query($q) or die("Problem with the query: $q<br>" . mysql_error());
   }
}

 

Anything jump out at you? Thanks for your help.

[sanchez77]

Hey Premiso,

 

Thanks for your help. I am going to mark this as solved as the curl funciton is working like it's suppose to. Thank you very much!

 

My new problem is the way it is seperating the array and creating values for the SQL statement. Maybe another post.

 

Thanks Premiso

 

Cheers