how do i fread a url that requires a login?

[thadeak]

Hi guys,

I am new to the forum, so I would just like to say hello to everyone :)

A little background info about me:
I am a PHP developer and have been for almost 4 years, I am familiar with PHP and am pretty competent but I rarely get the chance to do the real complex stuff.

The problem:
I am reading the contents of a page using fopen/fread, this works a treat, the problem is that the page I will need to read requires logging in first. Basically what I am doing is reading a page, and then finding all the images in that page and creating an array of images. The problem is that the page is on a "profile" type site and requires logging in to access the page. It's my profile and I can log in to the site as normal, but even after logging in and then I try and run the fopen script it still won't access the page.
The script is hosted on my own site and is trying to read the page from another site.
The way the profile site works is that you need to log in via a standard username/password form and it creates a cookie.

My question is, is there any possible way I can access the page?

I really appreciate all your time and help.

I hope someone has some ideas.

Cheers guys

Deak

[michaellunsford]

shove in the right direction:

[a href=\"http://us3.php.net/curl\" target=\"_blank\"]http://us3.php.net/curl[/a]

[thadeak]

Thanks Michael, that seems to be a shove in the right direction :) thanks man.

however, it has opened a whole new can of worms. something new for me to learn :)


i will let you know how i got on.

thanks again.

Deak

[thadeak]

After some 4 hours of reading up on cURL and trying out different examples, I finally cracked it :)
I won't be beaten, lol.

The info on php.net is...well over whelming to say the least, but it did get me started in the right direction. Thanks to Michael :)

and all the examples on the net semi worked. But they weren't giving the results I wanted.

I gradually got closer when i was able to sign in to the login page, but then couldn't access another "secure" page. This lead me to think the cookie wasn't being passed on, and sure enough that was the case.

lots of examples out there showed to set the cookieJar with a value like this "\tmp\mycookie" but this was giving me "Object moved to here." which is not what i wanted.

i ended up just using "mycookie" as the value for CURLOPT_COOKIEJAR/CURLOPT_COOKIEFILE, which places the cookie in the same directory as the script. This worked !!!! YAY, now I am betting that the dir that holds the script and the cookie is going to have to be writable when I upload it. I have never worked with cookies before, so my knowledge is very limited.

i created a dir called "tmp" in the root of the script and tried setting the cookie value to "\tmp\mycookie" but for whatever reason that didn't work. It's now 4:30am and I am just glad i got something working.

Anyway here is my code. You will need to replace the $url_login with the URL to the login page and the $url_view with the URL to the page you want to view once you have logged in. you will also need to put in the relevant info for $postData.

[code]
<?php
$cookieJar = "cookie";
//this should be set to whatever file on the computer you want to save and retrieve cookies to/from

$userAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051111 Firefox/1.5";
//you should set this to whatever specific browser you want to pretend to be. Usually this string is in your browsers Help->About window, but if not, google "user agent strings" and maybe you can figure out what exactly you want.

$postData = "usernam=youruserid&password=yourpassword";
//this is the post data that is sent with the request if you need it. This is just an example of a facebook login for mulka@umich.edu with password yarr.

$url_login = "http://www.myURL.com/login.php";
$url_view  = 'http://www.myURL.com/view.php';

$ch = curl_init(); //creates and initializes a curl session
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookieJar); //sets the file to save cookies in after recieving them from the page
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookieJar); //sets file to take cookies from to give to the site if needed
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent); //sets the user agent string to make it look like this is a real browser
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); //if a simple http redirect is recieved, setting this to 1 will cause curl to follow the url
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //this causes the data retrieved to be returned by the curl_exec function below instead of being printed
curl_setopt($ch, CURLOPT_URL, $url_login); //sets the url to request
curl_setopt($ch, CURLOPT_POST, 1); //this is a post request and not a get request like normal
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData); //the post data to be sent with the request
$data = curl_exec ($ch); //execute the request
curl_close ($ch); //close the curl session
unset($ch); //destroy the curl session.

$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookieJar);
curl_setopt($ch, CURLOPT_URL, $url_view);
$result = curl_exec ($ch);
curl_close ($ch);

echo $result;
?>
[/code]

I hope this helps someone.

wow, you do learn soemthing new everyday :)

good night and good luck

Deak

*be warned, curl is also used in linux as a command line.