Jump to content

Multiple pages with multi cURL?


MySQL_Narb

Recommended Posts

From the PHP manual page, it looks like you can only set the cURL options for all the handles you're including once. So it's like the multiple cURL sessions will only do what you told them to before you added them to the multi_curl handle.

 

But what if I needed EVERY (I want them all to do the same thing!) handle I added to the multi_handle to execute more than one page?

 

If the above isn't put into words properly, how do you go about doing something like this in multi_curl?

 

 

curl_setopt($currenthandle, CURLOPT_URL, 'http://www.google.com');
curl_exec($currenthandle);
curl_setopt($currenthandle, CURLOPT_URL, 'http://www.pageineedtopostto.com');
curl_setopt($currenthandle, CURLOPT_POST, true);
curl_setopt($currenthandle, CURLOPT_POSTFIELDS, array('param' => 'value');
curl_exec($currenthandle);
Link to comment
Share on other sites

EDIT: The below problem seems to be regarding XAMPP. I tried this code on my webhost, and it worked just fine. Weird.

 

Because this forum has an annoying policy regarding editing your posts, I guess I'll make another post.

 

I'm using the following cURL class: http://paste2.org/4kZz3pUg

 

Whenever I run one session, cURL works fine and the requested results load instantly. But if I add more than one (even if it's 2) session, my page never loads. The favicon is just the chrome loading symbol.

 

So this loads instantly/works:

<?php
 
class test {
    private $url = 'http://www.google.com';
    private $curl;
    
    function __construct(CURL $curl) {
        $this->curl = $curl;
        
        $opts = array(
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_FOLLOWLOCATION => true
        );
        
        $curl->addSession($this->url, $opts);
        
        $results = '<textarea>'.$curl->exec().'</textarea>';
        $curl->close();
        
        echo $results;
    }
}
 
?>

This does not work. The page never loads:

<?php
 
class test {
    private $url = 'http://www.google.com';
    private $curl;
    
    function __construct(CURL $curl) {
        $this->curl = $curl;
        
        $opts = array(
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_FOLLOWLOCATION => true
        );
        
        $curl->addSession($this->url, $opts);
        $curl->addSession($this->url, $opts);
        
        $results = '<textarea>'.$curl->exec().'</textarea>';
        $curl->close();
        
        echo $results;
    }
}
 
?>

I'm running XAMPP 

Edited by MySQL_Narb
Link to comment
Share on other sites

If you add more than one session (multiple addSession() calls), $curl->exec()  will return an array of results, so you need to loop through it.

 

$curl->addSession($this->url, $opts);
$curl->addSession($this->url, $opts);
$results = $curl->exec();
foreach ($results as $result) {
  echo '<textarea>'.$result.'</textarea>';
}
Also you may need to set CURLOPT_SSL_VERIFYPEER => false in your $opts array to ignore ssl cert validation requirements..dunno what urls you are actually using but google.com usually redirects to https.

 

Also..if you're trying to cURL google.com twice in a row like that, they more than likely are treating your 2nd hit like a bot since it's happening so quickly and you aren't even setting a user agent or anything (you can set the user agent string with CURLOPT_USERAGENT). They could be doing any number of things in that case, and your code doesn't show that you are doing anything to capture and report headers or errors or nothin'. If you do the above changes and it still doesn't work, try it on other urls.

 

If it still doesn't work, add these to your $opts array:

 

CURLOPT_HEADER => true,
CURLOPT_TIMEOUT => 20,
CURLOPT_CONNECTTIMEOUT => 120,
CURLOPT_MAXREDIRS => 10
Then change your code to be something more like this:

 

$curl->addSession($this->url, $opts);
$curl->addSession($this->url, $opts);
$results = $curl->exec();
$errors = $curl->error();
$headers = $curl->info();
foreach ($results as $i => $result) {
  echo "error: ".$errors[$i]."<br/>";
  echo "http code: ".$headers[$i]['http_code']."<br/>";
  echo '<textarea>'.$result.'</textarea><br/>';
}
This should hopefully give you some clues as to what's going wrong. Maybe the target server is slow and you have to up the timeout or somethin'.
Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.