Getting page source using curl

[Uruviel]

I want to get the source of a page so I can parse it and use it in my application. I always get the "max redirects reached" error although the site works fine in my browser.

 

The code I use:

<?php
$userAgent = "IE 7 - Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)";
$target_url = "http://landofconfusion.freedkp.org/frx/listmembers.php?show=all";

$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch, CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$html = curl_exec($ch);
if (!$html) {
echo "
cURL error number:" .curl_errno($ch);
echo "
cURL error:" . curl_error($ch);
exit;
}

 

The error I receive:

cURL error number:47 cURL error:Maximum (20) redirects followed

 

When you go to the site, you'll get this annoying popup like thing, it's just a div covering the page or something alike, but the full source is there. Yet I can't seem to get the source of that page so I can parse it.

 

Any idea what could be causing the problem?

 

[Deoctor]

if u want to get the source of the page us this curl tool which i have done long backk..

i didnt tested with your site yet..

 

<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.google.co.in/");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$output = curl_exec($ch);

$fh = fopen("output.txt", 'w');
fwrite($fh, $output);
?>

[gizmola]

I can't 100% guarantee you this will fix your issue, but i can say that the site wants to set a cookie, so you might try adding a cookie jar and see if that stops the runaway redirects.

[Uruviel]

I can't 100% guarantee you this will fix your issue, but i can say that the site wants to set a cookie, so you might try adding a cookie jar and see if that stops the runaway redirects.

 

Yep, it was as simple as that. Thanks.