[SOLVED] How do I scan a page to get data off of it.

[jeffjohnvol]

Hi, I checked the FAQ/Code Snippet Repository but did not see this.  I am working on a block in php-nuke (nuke-evolution to be exact) and I need to scan the page: http://lakeinfo.tva.gov/htbin/lakeinfo?site=WTH&DataType=All&submit=View+info  .  The data I'm looking for (the curent lake level) varies position, so I'm looking for an easy way to load the page into an array that I can parse.

 

Is there an easy way to do this?

 

Thanks.

Jeff

[cooldude832]

You can do it, but consistency is what your goal is, you will need something like

<?php
$data = file_get_contents("http://lakeinfo.tva.gov/htbin/lakeinfo?site=WTH&DataType=All&submit=View+info");
$data = strip_tags($data,"<td>");
$pattern = ">(??<t>[^<]*))";
$replacement "<td>";
$data = eregi_replace($pattern,$replacement,$data);
$bits = explode("<td>",$data);
print_r($bits);
?>

 

See what that does for you, the regex pattern might be off just found it

[Psycho]

Get a "screen scraper" - there are some free code snippets/classes you can get. Then you need to analyze the page and come up with rules to determine how to parse the information you want. However, if the site changes how the data is displayed your script will fail.

 

I'n not sure exactly what data from that page you are looking for. But, let's assume it is the "Upstream Elevation" under the "Observed" heading. based upon a quick scan of the page here is how I would extract that data:

 

1. Find the line with "< h2 align="left">Observed</ h2>"

2. Find the first table immediately following the above

3. In the 2nd row extract the text from the 2nd TD - this is the "date"

4. Starting with the 3rd row to the end of the table get the text in the 2nd (time) and 4th (level) TDs

[cooldude832]

this worked for me

<?php
$data = file_get_contents("http://lakeinfo.tva.gov/htbin/lakeinfo?site=WTH&DataType=All&submit=View+info");
$data = strip_tags($data,"<td>");
$pattern = "<[^>]*>";
$replacement = "<td>";
$data = eregi_replace($pattern,$replacement,$data);
$bits = explode("<td>",$data);

$level = array();
$place = 72;
while(!strstr($bits[$place],"* discharge reported at end of hour")){
$time = trim(str_replace(" ","",$bits[$place]));
$place = $place+4;
$level[$time]['Upstream'] = trim($bits[$place]);
$place = $place+4;
$level[$time]['Downstream'] = trim($bits[$place]);
$place = $place+4;
$level[$time]['Discharge'] = trim($bits[$place]);
$place = $place+4;
}
print_r($level);
?>

 

outputs:

Array

(

    [1am] => Array

        (

            [upstream] => 1,935.96

            [Downstream] => 1,647.90

            [Discharge] => 0

        )

 

    [2am] => Array

        (

            [upstream] => 1,935.97

            [Downstream] => 1,647.77

            [Discharge] => 0

        )

 

    [3am] => Array

        (

            [upstream] => 1,935.98

            [Downstream] => 1,647.65

            [Discharge] => 0

        )

 

    [4am] => Array

        (

            [upstream] => 1,935.98

            [Downstream] => 1,647.54

            [Discharge] => 0

        )

 

    [5am] => Array

        (

            [upstream] => 1,935.98

            [Downstream] => 1,647.43

            [Discharge] => 0

        )

 

    [6am] => Array

        (

            [upstream] => 1,935.98

            [Downstream] => 1,647.90

            [Discharge] => 10

        )

 

    [7am] => Array

        (

            [upstream] => 1,935.98

            [Downstream] => 1,650.10

            [Discharge] => 1,436

        )

 

    [8am] => Array

        (

            [upstream] => 1,935.97

            [Downstream] => 1,650.09

            [Discharge] => 1,438

        )

 

)

 

[jeffjohnvol]

cooldude832 & mjdamato, thanks a bunch for the quick reply.  I will play with it tonight and will report back.  I appreciate the detail in which you spelled it out for me.  You saved me quite a bit of work!

 

Thanks.

Jeff

[jeffjohnvol]

I haven't had a chance to test this (wife has me finishing basement before mother arrives ), but I'm going to mark it as solved.

 

Thanks again for the help.  This site is a great resource.  I hope to be able to contribute someday, but it may take awhile, lol.