Jump to content

Curl loop to fetch multiple pages - running cURL-Multi


Recommended Posts

good day dear community,


i am workin on a Curl loop to fetch multiple pages:


i have some examples - and a question:


Example: If we want to get information from 3 sites with CURL we can do it like so:


$list[1] = "http://www.example1.com";
$list[2] = "ftp://example.com";
$list[3] = "http://www.example2.com";


After creating the list of links we should initialize the cURL multi handle and adding the cURL handles.


$curlHandle = curl_multi_init();
for ($i = 1;$i <= 3; $i++)
$curl[$i] = addHandle($curlHandle,$list[$i]);



Now we should execute the cURL multi handle retrive the

content from the sub handles that we added to the cURL multi handle.


for ($i = 1;$i <= 3; $i++)
$text[$i] =  curl_multi_getcontent ($curl[$i]);
echo $text[$i];

In the end we should release the handles from the cURL multi handle by calling curl_multi_remove_handle and close the cURL multi handle!


If we want to another Fetch of sites with cURL-Multi - since this is the most pretty way to do it!

Well I am not sure bout the string concatenation. How to do it - Note I want to fetch several hundred pages:

see the some details for this target-server sites - /(I have to create a loop over several hundred sites).


    * siteone.example/?show_subsite=9009

    * siteone.example/?show_subsite=9742

    * siteone.example/?show_subsite=9871


.... and so on and so forth Question: How to appy this loop into the array of the curl-multi?



* Multi interface in PHP with curl  *
* Requires PHP 5.0, Apache 2.0 and  *
* Curl         *
* Writen By Cyborg 19671897         *
* Bugfixed by Jeremy Ellman         *

$urls = array(

$mh = curl_multi_init();

foreach ($urls as $i => $url) {
       curl_setopt($conn[$i],CURLOPT_RETURNTRANSFER,1);//return data as string 
       curl_setopt($conn[$i],CURLOPT_FOLLOWLOCATION,1);//follow redirects
       curl_setopt($conn[$i],CURLOPT_MAXREDIRS,2);//maximum redirects
       curl_multi_add_handle ($mh,$conn[$i]);

do { $n=curl_multi_exec($mh,$active); } while ($active);

foreach ($urls as $i => $url) {





I look forward to your ideas.



Link to comment
Share on other sites


This topic is now archived and is closed to further replies.

  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.