Uruviel Posted February 12, 2010 Share Posted February 12, 2010 I want to get the source of a page so I can parse it and use it in my application. I always get the "max redirects reached" error although the site works fine in my browser. The code I use: <?php $userAgent = "IE 7 - Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)"; $target_url = "http://landofconfusion.freedkp.org/frx/listmembers.php?show=all"; $ch = curl_init(); curl_setopt($ch, CURLOPT_USERAGENT, $userAgent); curl_setopt($ch, CURLOPT_URL,$target_url); curl_setopt($ch, CURLOPT_FAILONERROR, true); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); curl_setopt($ch, CURLOPT_AUTOREFERER, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER,true); curl_setopt($ch, CURLOPT_TIMEOUT, 10); $html = curl_exec($ch); if (!$html) { echo " cURL error number:" .curl_errno($ch); echo " cURL error:" . curl_error($ch); exit; } The error I receive: cURL error number:47 cURL error:Maximum (20) redirects followed When you go to the site, you'll get this annoying popup like thing, it's just a div covering the page or something alike, but the full source is there. Yet I can't seem to get the source of that page so I can parse it. Any idea what could be causing the problem? Quote Link to comment Share on other sites More sharing options...
Deoctor Posted February 12, 2010 Share Posted February 12, 2010 if u want to get the source of the page us this curl tool which i have done long backk.. i didnt tested with your site yet.. <?php $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, "http://www.google.co.in/"); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_BINARYTRANSFER, true); $output = curl_exec($ch); $fh = fopen("output.txt", 'w'); fwrite($fh, $output); ?> Quote Link to comment Share on other sites More sharing options...
gizmola Posted February 12, 2010 Share Posted February 12, 2010 I can't 100% guarantee you this will fix your issue, but i can say that the site wants to set a cookie, so you might try adding a cookie jar and see if that stops the runaway redirects. Quote Link to comment Share on other sites More sharing options...
Uruviel Posted February 12, 2010 Author Share Posted February 12, 2010 I can't 100% guarantee you this will fix your issue, but i can say that the site wants to set a cookie, so you might try adding a cookie jar and see if that stops the runaway redirects. Yep, it was as simple as that. Thanks. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.