Jump to content

Problem with Curl library, why response is not the same as in browser?


Radek_1
Go to solution Solved by tobbek,

Recommended Posts

Hello,

 

I wanted to download flight from Ryanair page.

I've checked all headers which are sended when browser "create" request.

 

It looks like this:

I'm entering at https://www.bookryanair.com/SkySales/Booking.aspx

In headers I can see, that theres two GET requests to the following pages:

1) Booking.aspx (here is created session ID)
2) Search.aspx (here is generated and assigned _VIEWSTATE as hidden field in form)

 

Until now, everything what I'm trying to do by curl is the same as in browser (responses are the same).

Now I'm complete form and data are sended by POST to:

3) Search.aspx

It uses the above mentioned SessionID and _VIEWSTATE, the rest of the fields I fill as in request from the browser.

In browser I can see flights, while when I'm trying to generate it by curl, I have the same result as in first GET reuqest to Search.aspx (2) (no data about flight, like empty requets).


Code from my last request to Search.aspx (3):

$url = "https://www.bookryanair.com/SkySales/Search.aspx";

$fields = array(
'_EVENTTARGET' => 'SearchInput$ButtonSubmit', 
'_EVENTARGUMENT' => '',
'_VIEWSTATE' => $viewstate,
'formaction' => 'Search.aspx',
'errorlist' => '',
'SearchInput$IsFlexible' => 'on',
'SearchInput$TripType' => 'RoundTrip',
'SearchInput$Orig' => 'WMI',
'SearchInput$Dest' => 'NYO',
'SearchInput$DeptDate' => '06/05/2014',
'SearchInput$RetDate' => '10/05/2014',
'SearchInput$PaxTypeADT' => '1',
'SearchInput$PaxTypeCHD' => '0',
'SearchInput$PaxTypeINFANT' => '0');

foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
					'Accept: text/html, */*; q=0.01',
                                        'Content-Type: application/x-www-form-urlencoded; charset=UTF-8',
                                        'Connection: keep-alive',
					'Cache-Control: no-cache',
					'Pragma: no-cache',
					'X-Requested-With: XMLHttpRequest'
                                            ));
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0');
curl_setopt($ch, CURLOPT_HEADER,1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_REFERER, 'https://www.bookryanair.com/SkySales/Booking.aspx');
curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE); 
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$fields_string);

$result = curl_exec($ch);

curl_close($ch)

$viewstate is from request to Search.aspx (2) and retrieved from the hidden field generated _VIEWSTATE.

Cookie_file - place with stored cookie.txt and sessionId.

What am I doing wrong? Is the problem in the communication that uses https protocol?

Thanks in advance.

Link to comment
Share on other sites

1) where is COOKIE_FILE defined?

 

2) you may wanna try setting

 

curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, '1');
3) since it's an https location, you're going to have to set a handful of curl settings that either deal with ssl authentication, or ignore it. The *easier* (though NOT recommended, except to narrow down the issue) method is to attempt to ignore it by setting CURLOPT_SSL_VERIFYPEER to false. If all is said and done and this is what finally makes it work, do NOT do this. Instead, work on using the correct curl options to validate ssl certs. If you simply turn off validation, this is what leads to man-in-the-middle attacks.
Link to comment
Share on other sites

  • 2 weeks later...

josh, thank you for your answser.

I've tired things you suggested but nothing happen :(

 

I've turn on CURLOPT_FOLLOWLOCATION and also turn off CURLOPT_SSL_VERIFYPEER but still it's give me same results.

 

COOKIE_FILE is definied at the beginning of the file

define("COOKIE_FILE", "cookie.txt");

Any other ideas how I can make it works?

Link to comment
Share on other sites

  • 2 weeks later...
  • Solution

I got it to work when I removed $fields_string altogether. Instead I pasted the POST I got from firebug (in Firefox), which you can see after you do a search on the ryanair website.

curl_setopt($ch, CURLOPT_POSTFIELDS,'__EVENTTARGET=SearchInput%24ButtonSubmit&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPDwUBMGRkwiC%2BdiO7YHrN9X8OU5GYY7FHcEU%3D&formaction=Search.aspx&errorlist=&SearchInput%24IsFlexible=on&SearchInput%24TripType=RoundTrip&SearchInput%24Orig=NYO&SearchInput%24Dest=WMI&SearchInput%24DeptDate=17%2F05%2F2014&SearchInput%24RetDate=22%2F05%2F2014&SearchInput%24PaxTypeADT=1&SearchInput%24PaxTypeCHD=0&SearchInput%24PaxTypeINFANT=0');
Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.