doncoglioni

That's the one. Thank you

I'd be absolutely delighted to, as soon as I get one tiny thing fixed with it - please do give my other (short) post a read; I'd love your help. It's right here ---> http://www.phpfreaks.com/forums/index.php/topic,188902.0.html Ooh, and for the benefit of us all: how do you get the fancy colored code? Mine's all greyscale Thanks again

Hey everyone I'm using cURL with PHP to automate some tasks online, basically logging into a website and going through various pages. I don't want to see any output on the screen when I run my .php script, basically I just need to "fetch" stuff and save it to disk. It functions perfectly, but every page I "visit" with cURL gets printed to my web browser, and it's very annoying! So is there a way to run cURL WITHOUT printing the source of the page it's fetching? I'll post my code, it's probably a silly mistake: $ch=curl_init(); curl_setopt($ch, CURLOPT_URL, "https:\\www.secure.site.com\login.php"); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt'); curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt'); curl_setopt($ch, CURLOPT_MAXREDIRS, 4); curl_setopt($ch, CURLOPT_FOLLOWLOCATION,0); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, 'username=user&password=secret'); curl_setopt($ch, CURLOPT_TIMEOUT, 200); curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9'); curl_setopt($ch, CURLOPT_REFERER, 'https:\\www.secure.site.com\index.php'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0); $buffer = curl_exec($ch); curl_close($ch); Thanks, everyone!

Thanks, guys. I found out the problem was that I wasn't following the "correct" login procedure for my website. On further examination, I noticed it doesn't use cookies to keep track of its visitors, and that if you go back to the home page, you're logged out automatically! So I just got the HTTPHeaders extension for Firefox, and copied all the headers as I moved through every page (from home page -> login -> welcome page -> account page -> my target page) and now I have 4 sets of cURL commands with the various headers going between those pages.

I thought I'd also post my code so you can see what's going on, and what's going wrong (in comments!) #### //Logging into the secure site, just passing username/password, referer and useragent. #### $ch=curl_init(); curl_setopt($ch, CURLOPT_URL, "https://secure.site.com/page1_userlogin.php"); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); curl_setopt($ch, CURLOPT_COOKIEJAR, 'C:\cookies.txt'); curl_setopt($ch, CURLOPT_COOKIEFILE, 'C:\cookies.txt'); curl_setopt($ch, CURLOPT_MAXREDIRS, 4); curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, 'username=abcdefg&password=1234567'); curl_setopt($ch, CURLOPT_TIMEOUT, 120); curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9'); curl_setopt($ch, CURLOPT_REFERER, 'http://secure.site.com/welcome_page.php'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0); $buffer = curl_exec($ch); curl_close($ch); #### // Now I'm logged in, I should be able to access other secure pages (to parse them with my parser!) #### #### // BUT I CAN'T - I parse the standard "not logged in" page instead :'( #### include("phpHTMLParser.php"); $parser = new phpHTMLParser(file_get_contents("https://secure.site.com/page2_this_is_the_target.php")); $HTMLObject = $parser->parse(); $HTMLObject = $parser->parse_tags(array("div")); $HTMLObject->output(); ?>

cunoodle2; you're an absolute genius. I have no idea what I was doing wrong, but now my script is logging me in to the website. HOWEVER - all that happens is the "successful login" page is displayed, and if I try to access any further pages within the member's area, it acts as if I'm not logged in at all. It's as if it's only logging me in for a second, then immediately logging me back out again. Is there a reason behind this that you might think of? Thank you, either way! Thank you very much.

Thank you for the reply. Just noticed that cURL seems ideal -- I found this, it might be what I'm looking for. But any suggestions, please let me know: http://www.wagerank.com/2007/how-to-submit-forms-with-php/

Hi all! I have a script which analyses HTML documents and summarises some information for them, related to my work. The script itself works fine, if I have access to the HTML pages concerned. However, most of the ones I need are only accessible when I log in to a (public) website with a username and password field. So my question is this: can a PHP script automatically "enter" the username and password, and log into the website, then (remaining authenticated for that session) fetch the protected pages that I need, just as if a human was sitting doing the same thing? No captchas are involved. Obviously I would need some information regarding the target page's username/password form and method of submission... but how would I go about doing this, generally? My sincere thanks for any help.