Problem with Curl library, why response is not the same as in browser?

[Radek_1]

Hello,

 

I wanted to download flight from Ryanair page.

I've checked all headers which are sended when browser "create" request.

 

It looks like this:

I'm entering at https://www.bookryanair.com/SkySales/Booking.aspx

In headers I can see, that theres two GET requests to the following pages:

1) Booking.aspx (here is created session ID)
2) Search.aspx (here is generated and assigned _VIEWSTATE as hidden field in form)

 

Until now, everything what I'm trying to do by curl is the same as in browser (responses are the same).

Now I'm complete form and data are sended by POST to:

3) Search.aspx

It uses the above mentioned SessionID and _VIEWSTATE, the rest of the fields I fill as in request from the browser.

In browser I can see flights, while when I'm trying to generate it by curl, I have the same result as in first GET reuqest to Search.aspx (2) (no data about flight, like empty requets).


Code from my last request to Search.aspx (3):

$url = "https://www.bookryanair.com/SkySales/Search.aspx";

$fields = array(
'_EVENTTARGET' => 'SearchInput$ButtonSubmit', 
'_EVENTARGUMENT' => '',
'_VIEWSTATE' => $viewstate,
'formaction' => 'Search.aspx',
'errorlist' => '',
'SearchInput$IsFlexible' => 'on',
'SearchInput$TripType' => 'RoundTrip',
'SearchInput$Orig' => 'WMI',
'SearchInput$Dest' => 'NYO',
'SearchInput$DeptDate' => '06/05/2014',
'SearchInput$RetDate' => '10/05/2014',
'SearchInput$PaxTypeADT' => '1',
'SearchInput$PaxTypeCHD' => '0',
'SearchInput$PaxTypeINFANT' => '0');

foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
					'Accept: text/html, */*; q=0.01',
                                        'Content-Type: application/x-www-form-urlencoded; charset=UTF-8',
                                        'Connection: keep-alive',
					'Cache-Control: no-cache',
					'Pragma: no-cache',
					'X-Requested-With: XMLHttpRequest'
                                            ));
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0');
curl_setopt($ch, CURLOPT_HEADER,1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_REFERER, 'https://www.bookryanair.com/SkySales/Booking.aspx');
curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE); 
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$fields_string);

$result = curl_exec($ch);

curl_close($ch)

$viewstate is from request to Search.aspx (2) and retrieved from the hidden field generated _VIEWSTATE.

Cookie_file - place with stored cookie.txt and sessionId.

What am I doing wrong? Is the problem in the communication that uses https protocol?

Thanks in advance.

[.josh]

1) where is COOKIE_FILE defined?

 

2) you may wanna try setting

 

curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, '1');

3) since it's an https location, you're going to have to set a handful of curl settings that either deal with ssl authentication, or ignore it. The *easier* (though NOT recommended, except to narrow down the issue) method is to attempt to ignore it by setting CURLOPT_SSL_VERIFYPEER to false. If all is said and done and this is what finally makes it work, do NOT do this. Instead, work on using the correct curl options to validate ssl certs. If you simply turn off validation, this is what leads to man-in-the-middle attacks.

[Radek_1]

josh, thank you for your answser.

I've tired things you suggested but nothing happen

 

I've turn on CURLOPT_FOLLOWLOCATION and also turn off CURLOPT_SSL_VERIFYPEER but still it's give me same results.

 

COOKIE_FILE is definied at the beginning of the file

define("COOKIE_FILE", "cookie.txt");

Any other ideas how I can make it works?

[tobbek]

Did you get any solution yet?

 

/Toby

[tobbek]

I got it to work when I removed $fields_string altogether. Instead I pasted the POST I got from firebug (in Firefox), which you can see after you do a search on the ryanair website.

curl_setopt($ch, CURLOPT_POSTFIELDS,'__EVENTTARGET=SearchInput%24ButtonSubmit&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPDwUBMGRkwiC%2BdiO7YHrN9X8OU5GYY7FHcEU%3D&formaction=Search.aspx&errorlist=&SearchInput%24IsFlexible=on&SearchInput%24TripType=RoundTrip&SearchInput%24Orig=NYO&SearchInput%24Dest=WMI&SearchInput%24DeptDate=17%2F05%2F2014&SearchInput%24RetDate=22%2F05%2F2014&SearchInput%24PaxTypeADT=1&SearchInput%24PaxTypeCHD=0&SearchInput%24PaxTypeINFANT=0');