Jump to content

Use CURL to read website and load fields? website automation?


cgchris99

Recommended Posts

Should I use CURL or wget to website info and preload the necessary fields?

I have never done this type of programming (ie CURL/wget)

I need it to access the site, log me in and go to a certain page and extract the information and store it  in my MySQL table.

Are there any examples on how to code something like this. Not the table work but the CURL/wget and filling in the fields.

Thanks for any advice.
Link to comment
Share on other sites

here's the first part from the manuel -- some additions to help you along. You'll probably need to make one post for the username, password, etc. Then make a second request for your data. The "CURLOPT_RETURNTRANSFER" part says you want to return the page queried to a variable. If you're a good curler, you can probably figure it out pretty quickly with the following example and links:

example from: http://usphp.com/manual/en/function.curl-exec.php
all curl options: http://usphp.com/curl_setopt

[code=php:0]<?php
// create a new curl resource
$ch = curl_init();

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.example.com/"); //full URL of your query
curl_setopt($ch, CURLOPT_HEADER, 0); //don't display or store the header information returned
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //curl_exec() will returns the page instead of true or false
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt"); //store incoming cookies to this file
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookiex.txt"); //retrieve cookies asked for by the site from this file
curl_setopt($ch, CURLOPT_POST, 1); //we're using the "POST" method
curl_setopt($ch, CURLOPT_POSTFIELDS, "user=myusername&password=mypassword"); //post these fields

// grab URL and pass it to the browser
$var = curl_exec($ch);

// close curl resource, and free up system resources
curl_close($ch);

echo $var;
?>[/code]
Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.