PHP cURL Crawler

[cordoprod]

Hi.

I am making a crawler with cURL.

What I want to do is to search in a DB, and get it from GET.

 

Example:

Age: 14 to 16 outputs agefrom=14&ageto=16

 

And then, I want to use cURL to search and get how many db lines there are in the age beetween 14 and 16, and then visit all the search result pages.

 

Also, additional i want to see how many lines there are, and more specific if you understand.

[Maq]

What are you asking from us?  Do you want someone to do it for you or do you have a problem with your current script?

[cordoprod]

Both, kind of...

I just know how to get the page... I need further help.

 

My code so far:

/**
* Get a web file (HTML, XHTML, XML, image, etc.) from a URL.  Return an
* array containing the HTTP server response header fields and content.
*/
function get_web_page( $url )
{
    $options = array(
        CURLOPT_RETURNTRANSFER => true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => true,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_USERAGENT      => "spider", // who am i
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
    );

    $ch      = curl_init( $url );
    curl_setopt_array( $ch, $options );
    $content = curl_exec( $ch );
    $err     = curl_errno( $ch );
    $errmsg  = curl_error( $ch );
    $header  = curl_getinfo( $ch );
    curl_close( $ch );

    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}

 

[cordoprod]

Any ideas?

[cordoprod]

I have a new one..

I just want to have a form like mentioned with e.g age..

And then output it in GET form, and then visit all the results.

 

Anyone knows how to do that?