Jump to content

Cron a Login and Page Grab


electricblue

Recommended Posts

Hiya everyone,

 

I'm currently running a cron job every hour or so to do a file() grab of a page on a remote server, and then manipulate the resulting HTML for my own use.  The problem is, I'd like to move the script on and have it actually log into an account on the server (via a username/password i've set), and grab the "members only" information from there.

 

Any ideas on how I can do it? I'm not looking for a completed code, just a point in the right direction on what I need to do :)

 

Thanks all!

Link to comment
https://forums.phpfreaks.com/topic/46588-cron-a-login-and-page-grab/
Share on other sites

Depends how the server authenticates.  First of all, you'll have to use curl for this job.  Anyways, if the server authenticates using sessions, it shouldn't be too much of a problem.  You'd connect to the login page with curl, and then use curl functions to submit the username and password, and then you'd wait for the response.  However, if the site uses cookies to keep you logged in, it may be a little more tough.  I'm pretty sure curl doesn't save the cookies that a server sends back to the script, so you'll have to parse the response headers yourself and save the cookies in an array, and then send those cookies back to the server when you connect again to get your data.

Thanks.  The server does use cookies to hold the session login, I tried it with curl before but yes, it wasn't holding the cookie, and didn't think of catching it and sending it back.

 

I've heard snoopy could do this easier, or am I just causing myself problems there? :)

I think cURL does handle cookies if you assign the cookie jar (a file) to hold it.

 

    curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
    curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');

 

It then posts the cookie data like a normal browser would.

 

monk.e.boy

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.