[SOLVED] Custom Built Radar Weather Tracker

[barryhayden]

I'm in the process of building a time lapsed weather radar system using info from the http://radar.weather.gov site. I need to check a file folder on their site once an hour to update the current weather radar images that will be used in my custom app. The reason I have to do it this way is because the file names change according to when the last image was taken. When the weather is nice it takes an image every 6 min., and if the weather is bad it takes an image every 4 min., so you can't predict what the next file name will be. That's why I need to pull the file info from the folder on a regular basis.

 

The code I have here will retrieve the file listings from the folder. The problem is that I get too much information! I just want to grab the image names and place them into an array so I can then go grab the images and update the folder on my server. Hopefully that makes sense. If you run the script I'm sure you'll see what I mean. And thanks in advance, I really appreciate the help.

 

$dir = "http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/";
$handle = @fopen($dir, "r");
if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 9999);
        echo $buffer;
    }
    fclose($handle);
}

[ignace]

$mydir = realdir('path/to/directory');
$dir = 'http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/';
$images = scandir($dir);
foreach ($images as $image) {
    if ('.' === $image[0]) continue;
    
    $fullurl = implode('/', array($dir, $image));
    $fullpath = implode(DIRECTORY_SEPARATOR, array($mydir, $image));
    if (!file_exists($fullpath)) {
        $fs = filesize($fullurl);
        $fh1 = fopen($fullpath, 'w');
        $fh2 = fopen($fullurl, 'r');
        
        fwrite($fh1, fread($fh2, $filesize), $filesize);
        fclose($fh1);
        fclose($fh2);
    }
}

[barryhayden]

Hi Ignace, thanks for the fast reply. If I run your script I get...

Fatal error: Call to undefined function realdir() in C:\xampp\htdocs\Franklin_Kansas\content\radar\Split_Gif\newTest.php on line 2.

 

I guess what I don't understand about the script that you came up with is what does "$mydir = realdir('path/to/directory');" do for the script?

[ignace]

Woeps sorry my bad should be realpath It returns an absolute path to your directory.

 

You can read more about it here: http://us.php.net/manual/en/function.realpath.php

[barryhayden]

I'm pretty new at trying to do anything like this with php, so now I'm really lost. I don't understand what your code is trying to do. Maybe I should go into a little more detail to clarify the goal. If you were to go to http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/ you would see a list of the last twenty or so radar images by UTC timecode. I actually only want to get say the most recent ten images to use.

 

Sitting beside my script (in the first post) is a folder called radarImages. Using that script gives me the information from the link above. I want to get the names of most recent ten images and step through them with something like this

$curImage = 'ImageFromList';
$i = 0;
if(!copy("http://radar.weather.gov/ridge/RadarImg/N0Z/" . $curImage, "radarImages/Kansas_City_Radar_" . $i . ".gif")){
echo("failed to copy file");
}

which will load and rename them into my radarImages folder so that I can use them in my app. And the last part of this is that the script will need to replace the current images in the folder with the new ones. Hopefully that explains all the small details.

 

I really appreciate your help with this. I've been frying my brain for a couple of days on it. And if it wouldn't be too much to ask, could you add a few notes. I really want to get an understanding behind the details of doing something like this so I won't have to ask questions in the future. Thanks again!

[ignace]

Here is my code again with comments

 

// absolute path to the local image directory
$mydir = realdir('path/to/directory');

// path to the remote images directory
$dir = 'http://radar.weather.gov/ridge/RadarImg/N0Z/EAX'; // fixed this, it had a trailing slash

// get all files (images) from the remote directory
$images = scandir($dir);

// go over each image
foreach ($images as $image) {
    
    // if $image starts with . ignore it (., .., .htaccess)
    if ('.' === $image[0]) continue;
    
    // 'absolute' path to the remote file (image)
    $fullurl = implode('/', array($dir, $image));
    
    // absolute path to the local file (or where it should be)
    $fullpath = implode(DIRECTORY_SEPARATOR, array($mydir, $image));
    
    // if the image does not yet exists, create it!
    if (!file_exists($fullpath)) {
        
        // get the size in bytes, we'll use this to read and write the total number of bytes
        $fs = filesize($fullurl);
        
        // file handle for the local image file (if it doesn't exist, mode 'w' creates it)
        $fh1 = fopen($fullpath, 'w');
        
        // file handle to the remote image file (read-only, write won't work unless the server is badly configured)
        $fh2 = fopen($fullurl, 'r');
        
        // write the data to $fh1 using the read data from $fh2 using a total length of $filesize bytes
        fwrite($fh1, fread($fh2, $filesize), $filesize);
        
        // close handles
        fclose($fh1);
        fclose($fh2);
    }
}

[barryhayden]

Thank you so much for adding the comments. It really helps to clarify things for me. And a very nice job on the script too! It's concise and it covers contingencies. But there is a slight problem with it.

 

I of course changed $mydir = realdir('path/to/directory'); to be $mydir = realpath('radarImages/'); and got these errors...

 

Warning: scandir(http://radar.weather.gov/ridge/RadarImg/N0Z/EAX) [function.scandir]: failed to open dir: not implemented in C:\xampp\htdocs\Franklin_Kansas\content\radar\Split_Gif\newTest.php on line 7

 

Warning: scandir() [function.scandir]: (errno 0): No error in C:\xampp\htdocs\Franklin_Kansas\content\radar\Split_Gif\newTest.php on line 7

 

Warning: Invalid argument supplied for foreach() in C:\xampp\htdocs\Franklin_Kansas\content\radar\Split_Gif\newTest.php on line 9

 

The last error I'm sure is coming because of the first two.

[barryhayden]

ignace, I searched on the web for (errno 0) and found this... This is because you're passing NULL (or empty string) to scandir.

[ignace]

Warning: scandir(http://radar.weather.gov/ridge/RadarImg/N0Z/EAX) [function.scandir]: failed to open dir: not implemented in C:\xampp\htdocs\Franklin_Kansas\content\radar\Split_Gif\newTest.php on line 7

 

'not implemented' means that the http protocol was not yet implemented for scandir(), opendir(), .. So you may want to look at other alternatives like FTP (http://us3.php.net/manual/en/book.ftp.php) or cURL (http://be.php.net/curl) I tried it locally and didn't work either altough allow_url_fopen = On

 

P.S. If you opt for FTP, once you have established a ftp connection use http://be.php.net/manual/en/function.ftp-nlist.php to get all files within the directory.

[barryhayden]

I don't have their ftp information so I don't think I can go that route. I know that this script works to get the information from the folder...

<?php
$dir = "http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/";
$handle = @fopen($dir, "r");

if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 9999);
        echo $buffer;
    }
    fclose($handle);
}
?>

I just need to figure out how to get the image names stripped out of the information and put into an array. Once that's done I can use a for loop and run this script...

<?php
$curImage = 'Image [0] from the array';
$i = 0;
if(!copy("http://radar.weather.gov/ridge/RadarImg/N0Z/" . $curImage, "radarImages/Kansas_City_Radar_" . $i . ".gif")){
echo("failed to copy file");
}
?>

If I can get it to that point I will be happy! But thanks for your help with this. I appreciate it.

[ignace]

I don't have their ftp information so I don't think I can go that route. I know that this script works to get the information from the folder...

<?php
$dir = "http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/";
$handle = @fopen($dir, "r");

if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 9999);
        echo $buffer;
    }
    fclose($handle);
}
?>

I just need to figure out how to get the image names stripped out of the information and put into an array. Once that's done I can use a for loop and run this script...

<?php
$curImage = 'Image [0] from the array';
$i = 0;
if(!copy("http://radar.weather.gov/ridge/RadarImg/N0Z/" . $curImage, "radarImages/Kansas_City_Radar_" . $i . ".gif")){
echo("failed to copy file");
}
?>

If I can get it to that point I will be happy! But thanks for your help with this. I appreciate it.

 

What does echo $buffer output?

[barryhayden]

echo $buffer outputs the same html page as if you went directly to...http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/ It looks identical.

[barryhayden]

Well, I'm getting a little closer. This script pulls out the file names in reverse order so I used an rsort to arrange the list like I want it.

<?
$matches = array();

preg_match_all("/(a href\=\")([^\?\"]*)(\")/i", get_text('http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/'), $matches);

rsort ($matches[2]);
foreach($matches[2] as $match){
echo $match . '<br>';
}

function get_text($filename){
$fp_load = fopen("$filename", "rb");
if ($fp_load){
	while (!feof($fp_load)){
		$content .= fgets($fp_load, 8192);
	}
	fclose($fp_load);
	return $content;
}
}
?>

I just want to be able to grab the last 15 images to use in my app. The problem I'm seeing now is if I insert this script into the mix...

for($i=0;$i<=15;$i++){
if(!copy("http://radar.weather.gov/ridge/RadarImg/N0Z/" . $match, "radarImages/kansasRadar_" . $i . ".gif")){
echo("failed to copy file");
}
}

I don't think it's actually grabbing the right files. Sometimes it looks like it grabs 4 or five of the exact same image. Any ideas on this that would make sure I'm getting the right images?

[thebadbad]

Here's a way of doing it. I modified your regular expression pattern, and use file_get_contents() instead of your get_text() function (does the same thing). http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/?C=N;O=D sorts the images to newest first.

 

<?php
$data = file_get_contents('http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/?C=N;O=D');
preg_match_all('~href="(EAX[^"]*)"~i', $data, $matches);
for ($i = 0; $i < 10; $i++) {
$src = "http://radar.weather.gov/ridge/RadarImg/N0Z/EAX/{$matches[1][$i]}";
if (!copy($src, "radarImages/kansasRadar_$i.gif")) {
	echo "Failed to copy file $src<br />";
}
}
?>

 

ignace's code didn't work because you don't have access to the file/folder structure on a remote site (that isn't yours). Would be a major security issue if you had.

[barryhayden]

thebadbad,

Beautiful piece of code. It worked perfectly! Thank you so much for your help! This thing was starting to drive me nuts. I would like to ask you one question though if I could. I'm planning on calling this script once an hour to update the images. Will they overwrite the current images in the directory? Or, will I need to delete the images in the directory and then call this script? And thanks again for your help, I really appreciate it!

[MatthewJ]

This might sound crazy... but why not just run the script in an hour and see what happens?

 

 

[thebadbad]

No worries. I'm pretty sure copy() tries to overwrite existing files, but as MatthewJ said, why not give it a try?

[barryhayden]

Sorry guys, I didn't mean to seem impatient. It's just so rainy throughout the midwest that I wasn't sure I'd even notice a difference. I copied the original files to another folder and ran the script again. It looks like it's overwriting the files just fine. Thanks thebadbad for your help with this. I really appreciate it. I'm marking this one solved!