Curl Login No Cookie

[bschultz]

I'm trying to login to a site, navigate a couple of pages deep, and then scrape a table.  I had all this working, but the site changed their login workflow...and I can't figure out what I need to do to make this work.

The form used in a browser to login has itself as the action page.  In a browser, you are automatically redirected to another page, that puts a random get variable at the end of the URL.

Here's the main login page:  https://www.premiereaffidavits.com/scripts/cgiip.exe/WService=webprime/login.html

When I run a standard curl login code...I only get the login page returned to me.  How can I get the redirected page (including the random GET variable at the end of the URL) that you get using a browser?  I see no javascript, or other ways for the action page to redirect to the second page you get in a browser.  What am I missing here?

I can generate login info if you need it.

Thanks.

Edited March 30, 2021 by bschultz

[gw1500se]

Perhaps if you posted your code (using the code icon (<>) in the menu and specified PHP) you might get an answer.

[bschultz]

<?php

$username = 'xxx';
$password = 'xxx';
$postdata = '?username='.$username.'&password='.$password;

$url = 'https://www.premiereaffidavits.com/scripts/cgiip.exe/WService=webprime/login.html';

$ch = curl_init(); 
curl_setopt ($ch, CURLOPT_URL, $url); 
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); 
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); 
curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_REFERER, $url); 
curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata); 
curl_setopt ($ch, CURLOPT_POST, 1); 
$result = curl_exec ($ch); 

echo $result;  

curl_close($ch);
?>

This only returns the login page...not the redirect page

[kicken]

You need to send the name of the button also.

Use the browsers dev tools or something like Fiddler to inspect all the requests being made and re-create the ones you need as closely as possible.

 

Edited March 30, 2021 by kicken

[bschultz]

Now I feel dumb...submit button!  Thank you!