Log in and collect data

[millercj]

I'm trying to come up with a real basic script to do the following below. I've never done something this back-end intensive so I'm looking for some pointers on the best way to do it.

 

The user will go to a page hosted on my site and will type a keyword into an input box.

After they hit submit, I'd like the script to log into another website (user name & password required), run a query in that sites search box using the keyword the user input on the original screen. I would then like it to return some elements from the HTML of the results page.

 

What is the best way to do all this?

[cags]

Using cURL will be the best way to achieve this. The exact steps required will vary depending on the exactly technology used by the target site/server. If it is a standard HTML/PHP driven site then it shouldn't be too complex at all. If it uses ASPX then things could be a lot more complex.

[millercj]

Ok, so I did some reading up on cURL today and I've got it successfully logging in but If i have the page that it loads display it says "This object has moved, click here to view". I'm assuming that this is some sort of security measure the site has in place...is there a way to get around it? my code is below:

 

<?PHP
curl_login('https://www.XXXXXXXXXXX.com/siteminderagent/forms/aelogin.fcc?TYPE=33554433&REALMOID=06-7401944f-341e-4ee7-abe4-1d2985a80e59&GUID=&SMAUTHREASON=0&METHOD=GET&SMAGENTNAME=TbFKJX2BHQWFkDQ8G72HsB1rD0Y2SC4o7TyRG8OmwvoNmNVyPRbY222iQkyp07n5&TARGET=-SM-HTTPS%3a%2f%2fwww%XXXXXXXXX%2ecom%2fRiskManagement%2fCostGuides%2fToMSB%2easp%3fSite%3dRCT','SMENC=ISO-8859-1&SMLOCALE=US-EN&target=HTTPS%3A%2F%2Fwww.XXXXXXXXXXX.com%2FRiskManagement%2FCostGuides%2FToMSB.asp%3FSite%3DRCT&smauthreason=0&smagentname=TbFKJX2BHQWFkDQ8G72HsB1rD0Y2SC4o7TyRG8OmwvoNmNVyPRbY222iQkyp07n5&postpreservationdata=&USER=XXXXXX&PASSWORD=XXXXXX&x=24&y=13','','off');

echo curl_grab_page('https://www.XXXXXXXX.com/RiskManagement/CostGuides/ToMSB.asp?Site=RCT','','off');

function curl_login($url,$data,$proxy,$proxystatus){
    $fp = fopen("cookie.txt", "w");
    fclose($fp);
    $login = curl_init();
    curl_setopt($login, CURLOPT_COOKIEJAR, "cookie.txt");
    curl_setopt($login, CURLOPT_COOKIEFILE, "cookie.txt");
    curl_setopt($login, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
    curl_setopt($login, CURLOPT_TIMEOUT, 40);
    curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
    if ($proxystatus == 'on') {
        curl_setopt($login, CURLOPT_SSL_VERIFYHOST, FALSE);
        curl_setopt($login, CURLOPT_HTTPPROXYTUNNEL, TRUE);
        curl_setopt($login, CURLOPT_PROXY, $proxy);
    }
    curl_setopt($login, CURLOPT_URL, $url);
    curl_setopt($login, CURLOPT_HEADER, TRUE);
    curl_setopt($login, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
    curl_setopt($login, CURLOPT_POST, TRUE);
    curl_setopt($login, CURLOPT_POSTFIELDS, $data);
    ob_start();     
    return curl_exec ($login); 
    ob_end_clean(); 
    curl_close ($login);
    unset($login);    
}                  

function curl_grab_page($site,$proxy,$proxystatus){
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
    if ($proxystatus == 'on') {
        curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
        curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
        curl_setopt($ch, CURLOPT_PROXY, $proxy);
    }
    curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
    curl_setopt($ch, CURLOPT_URL, $site);
    ob_start();      
    return curl_exec ($ch); 
    ob_end_clean();  
    curl_close ($ch);
}  
?>