Jump to content

file get contents with complicated (?) login first


muppet77

Recommended Posts

i am trying to do a "file get contents" from the following website
http://www.bet365.com/home/

It is necessary to input your account username and password here, and then I want to get the contents of this page
http://www.bet365.com/extra/en/results-and-archive/results/?Fromdate=13%2f09%2f2014+00%3a00&SearchPath=result&FixtureId=46827110&Period=99&ChallengeId=26278650&CompetitionId=999999&SportId=1&SportDesc=Soccer&Todate=14%2f09%2f2014+00%3a00&LanguageId=1&Zoneid=1

I have tried a couple of scripts but just can't seem to get it to work.

I have tried


$context = stream_context_create(array(
    'http' => array(
        'header'  => "Authorization: Basic " . base64_encode("$username:$password")
    )
));
$data = file_get_contents($url, false, $context);

echo $data;


with the username and password and url entered as variables.
but get the message

Warning: file_get_contents(http://www.bet365.com/extra/en/results-and-archive/results/?Fromdate=13%2f09%2f2014+00%3a00&SearchPath=result&FixtureId=46827110&Period=99&ChallengeId=26278650&CompetitionId=999999&SportId=1&SportDesc=Soccer&Todate=14%2f09%2f2014+00%3a00&LanguageId=1&Zoneid=1) [function.file-get-contents]: failed to open stream: HTTP request failed! in "myscript's address"  on line 13


Maybe i don't need to visit the homepage first to log in?
Any ideas please?
Thank you

apologies for the message being in code quotes, I wrote it in Word but then couldn't paste it, even using the special Word paste button.

Hope the message comes out ok.

Thank you.

 

Link to comment
Share on other sites

This site uses a cookie-based session system. If you inspect the cookies in your browser, you'll see one called “ASP.NET_SessionId”. This obviously carries the session ID.

 

So you need to send your credentials to their ASP.NET application, store the session cookie and then send it along with every request. This is way beyond the capablities of file_get_contents(). Use the cURL library instead.

 

Note that you'll need a basic understanding of how HTTP works.

  • Like 1
Link to comment
Share on other sites

There are literally hundreds of examples on Google. Why don't you look for them yourself and come back when you have a concrete question?

 

To get familar with the authentication procedure, it might be a good idea to set up a simple test application on your localhost. Make one log-in script and one script which requires a valid session. See if you can post your credentials to the log-in script and store the session cookie you get back. If that works, see if you can access the protected script with the stored cookie.

  • Like 1
Link to comment
Share on other sites

Search for something like “php curl login”. On the first page, there's already a Stack Overflow thread where this problem is discussed in great detail and with many examples.

 

The relevant cURL options are CURLOPT_POST and CURLOPT_POSTFIELDS for making the initial POST request. You'll also need CURLOPT_COOKIEJAR for writing the session cookie to a file and CURLOPT_COOKIEFILE for reading the cookie from that file.

Link to comment
Share on other sites

ok, had a look at those, thank you.

I get a blank page returned from this code:

//init curl
$ch = curl_init();

//Set the URL to work with
curl_setopt($ch, CURLOPT_URL, $url);

// ENABLE HTTP POST
curl_setopt($ch, CURLOPT_POST, 1);

//Set the post parameters
curl_setopt($ch, CURLOPT_POSTFIELDS, 'user='.$username.'&pass='.$password);

//Handle cookies for the login
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');

//Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
//not to print out the results of its query.
//Instead, it will return the results as a string return value
//from curl_exec() instead of the usual true/false.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

//execute the request (the login)
$store = curl_exec($ch);

//the login is now done and you can continue to get the
//protected content.

//set the URL to the protected file
curl_setopt($ch, CURLOPT_URL, $grab);

//execute the request
$content = curl_exec($ch);

curl_close($ch);


echo $content;

$username, $password are set as my login variables

and

$url = 'http://www.bet365.com';
$grab = 'http://www.bet365.com/extra/en/results-and-archive/results/?Fromdate=13%2f09%2f2014+00%3a00&SearchPath=result&FixtureId=46827110&Period=99&ChallengeId=26278650&CompetitionId=999999&SportId=1&SportDesc=Soccer&Todate=14%2f09%2f2014+00%3a00&LanguageId=1&Zoneid=1';
 

any suggestions please?

Link to comment
Share on other sites

You're not sending the cookies anywhere. And the entire log-in procedure looked more complicated to me than just sending the name and password to the site.

 

Like I said, do this step by step. Create a simple test script on your PC and try out cURL in a controlled environment before you jump to some live site.

Link to comment
Share on other sites

Link to comment
Share on other sites

I am afraid I have no idea what a simple test script would look like?

 

Like I explained in #4: One script for logging in and one script which checks the log-in status. The log-in script can be as simple as

<?php

session_start();

if ($_POST['name'] == 'foo' && $_POST['password'] == 'bar')
{
	$_SESSION['logged_in'] = true;
	echo 'Log-in successful';
}
else
{
	echo 'Log-in failed';
}

The point is that you should test and fix your cURL code before you go to the next step of using it on an actual site. It's not helpful to end up with some blank page and no idea what might be wrong.

 

 

 

 

 

 

This login page  requires javascript to be enabled in your browser. No way to use cURL.

 

Um, what? You do realize that the end result is always an HTTP request, right? Or did you think JavaScript performs some kind of magic?

Edited by Jacques1
Link to comment
Share on other sites

You send the request to the script that ajax would normally be submitting the login info to. It doesn't need to be an "ajax" request, unless they are specifically looking for the XMLHttpRequest header in which case you'd just need to add it to the CURL header to simulate an ajax request.

Something like:

curl_setopt($ch, CURLOPT_HTTPHEADER, array("X-Requested-With: XMLHttpRequest", "Content-Type: application/json; charset=utf-8"));
Edited by CroNiX
Link to comment
Share on other sites

Ok, how will you send an ajax request (if the script is required) without javascript?

 

An Ajax request is an HTTP request. Have you never opened the “network” tab of your browser to see what's going on behind the scenes?

 

It doesn't matter if the request comes from JavaScript or cURL or whatever. You might as well write it down by hand. The server doesn't know and doesn't care. It's all just a bunch of bytes in a bunch of IP packets.

Edited by Jacques1
Link to comment
Share on other sites

Like I explained in #4: One script for logging in and one script which checks the log-in status. The log-in script can be as simple as

 

<?php

session_start();

if ($_POST['name'] == 'foo' && $_POST['password'] == 'bar')
{
	$_SESSION['logged_in'] = true;
	echo 'Log-in successful';
}
else
{
	echo 'Log-in failed';
}
The point is that you should test and fix your cURL code before you go to the next step of using it on an actual site. It's not helpful to end up with some blank page and no idea what might be wrong.

 

 

 

 

 

 

 

 

 

 

Um, what? You do realize that the end result is always an HTTP request, right? Or did you think JavaScript performs some kind of magic?

Thanks. Unfortunately my knowledge is not that good.

I don't understand how that script logs in anywhere.

Really sorry!

Link to comment
Share on other sites

Jacques, who is talking about how the server treats the http post/get form? I am talking about that there is no way to make an ajax call without javascript and this could be easily checked (if js is enabled/disabled by php ( php is cross-side language))

 

CroNiX in the official docs of cURL, there is nothing about js and curl implementation. I don't think your example is possible to send data in this way by curl.

 

muppet77, what is your username and password. Send me a PM if you want it, to check if it's possible to use curl and lately I'll post the solution to the forum.         

Edited by jazzman1
Link to comment
Share on other sites

No offense, jazzman, but I think you're very confused.

 

The communication between the client and the server consists of exchanging IP packets (which contain the HTTP messages). The server doesn't know anything about the client except the IP address. It has absolutely no idea what the client did to generate the data: Could be Ajax, could be a “classical” request, could be cURL or any other tool. Hell, the client could have manually typed in the bytes with their keyboard. We don't know, and we cannot find out. All we know is that a particular IP address has sent us an HTTP request.

 

For the server, there's no such thing as “Ajax”. This is a client-side technique, and you have no way to detect or enforce it.

 

Think of the client as a black box: Data goes in, data goes out, but what happens inside the box is completely unknown. Is it a human user or a bot? Do they use a browser? If they use a browser, do they have JavaScript enabled? We have no idea.

Edited by Jacques1
Link to comment
Share on other sites

You are still continue to teach me how the server and client works. No problem, no offense is taken by me :) Now, I already know that there is not way php to detect if javascript is enabled / disabled in my browser when I make an ajax call to the server. So, back to the OP problem, here is the logic credentials:

#!/bin/bash

NAME='user'
PASS='pass'
referer='http%3a%2f%2fwww.bet365.com%2fxtra%2fen%2f'

#curl --cookie-jar cjar --output /dev/null 'https://members.bet365.com/Members/lp/default.aspx'

curl --cookie cjar --cookie-jar cjar 'https://members.bet365.com/Members/lp/default.aspx' --data "ctl00_AjaxScriptManager_HiddenField=&__EVENTTARGET=ctl00%24HeaderPlaceHolder%24HeaderControl%24Login%24Go&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPDwULLTE0OTk1NjUxNDBkZM%2F826rAqHRdNx1mZCnaiwnKJMB7&__PREVIOUSPAGE=b_rpb_GyZeP1TGPyzN2vQPsEv-IJbnLMMZcHunMJW9JgkSoJrqm6glz93dN4lmrcFcwnZGiTdwRvEiEWZLRm3YVm7wU1&__EVENTVALIDATION=%2FwEWDQKgifDsDwLBpdiBBQKF1rTBCwKg14akCAL%2FnP6QCgK6iKG3CgLx%2BNLkCwLk5KvOAwLBnamGCALUi53qCALxrtCaBwLNvuHaCQK24aRbEcr2XoX5okTIDQsKsnM8zQd6ypk%3D&ctl00%24HeaderPlaceHolder%24ctl00%24hidSessionTimestamp=&ctl00%24HeaderPlaceHolder%24ctl00%24hidSessionHashvalue=&ctl00%24HeaderPlaceHolder%24ctl00%24hidSessionTimeoutDuration=&ctl00%24HeaderPlaceHolder%24ctl00%24hidSessionAppPath=&ctl00%24HeaderPlaceHolder%24HeaderControl%24Login%24Username=${NAME}&ctl00%24HeaderPlaceHolder%24HeaderControl%24Login%24InitialPassword=Password&ctl00%24HeaderPlaceHolder%24HeaderControl%24Login%24ProtectedPassword=${PASS}&txtPassword=${PASS}&txtUserName=${NAME}&txtTKN=7EDA0805C01A4AD3AA96165DA59A4461000003&txtLH=0f740fa6750303c821740356981f2e1123e37676&txtTS=20140919204322079&txtType=2&txtSTKN=695060f20f99447ea7406c3d37a75491&txtScreenSize=1280+x+800&txtFlashVersion=11.2.202" --referer=${referer}

curl --cookie cjar --output ~/bet365.html 'http://www.bet365.com/extra/en/'

I'm not finding myself in login action using curl. Ideas instead of teaching each others what is AJAX? 

Edited by jazzman1
Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.