Jump to content

curl_multi_init and get multi page in php


tazeha

Recommended Posts

Hello

 

I write following code to get Multiple page of the this site: www.mobile.ir

This website have mobile price category .. this category have 42 navigation page.

I write following code with curl_multi_init and xpath to get data from Multiple page from category.

$ch1= curl_init ("http://www.mobile.ir/phones/prices.aspx");
$ch2 = curl_init ("http://www.mobile.ir/phones/prices.aspx?sort=date&dir=desc&brandid=0&terms=&duration=14&pagesize=50&price_from=-1&price_to=-1&provinceid=0&shopid=0&page=2");
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch1,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch1, CURLOPT_ENCODING, 'UTF-8');
curl_setopt($ch2, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch2,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
curl_setopt($ch2, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_ENCODING, 'UTF-8');

$mh = curl_multi_init();

//add the two handles
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);

$active = null;
//execute the handles
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}



$dom = new DOMDocument('1.0', 'utf-8');
libxml_use_internal_errors(true);
$dom->loadHTML($mrc);
libxml_clear_errors();
$xpath = new DOMXpath($dom);

$data = array();
$table_rows = $xpath->query('//table[@id=price_table]/tbody/tr'); // target the row (the browser rendered <tbody>, but actually it really doesnt have one)

if($table_rows->length <= 0) { // exit if not found
echo 'no table rows found';
exit;
}
$s = "st\xECna";
foreach($table_rows as $tr) { // foreach row
$row = $tr->childNodes;
if($row->item(0)->tagName != '<tr class="carHeader"> <td>نام کارخانه </td> <td>نام خودرو </td> <td>قیمت نمایندگی (ریال) </td> <td>قیمت بازار (ریال) </td> </tr>') { // avoid headers
$data[] = array(
'Name' => trim($row->item(2)->nodeValue),
'Price' => trim($row->item(4)->nodeValue),

);
}
}

echo '<pre>';
print_r($data); 

But don't display data ...

when I use curl_init() to get data from one page without problem working and displaying data (name and price) but when use curl_multi_init dont display I really need..

Link to comment
Share on other sites

Take mine and use code tags when providing code the next time.

<?php

ini_set('display_startup_errors',1);

ini_set('display_errors',1);

error_reporting(-1);

// create both cURL resources
$ch1 = curl_init();
$ch2 = curl_init();

// set URL and other appropriate options
curl_setopt($ch1, CURLOPT_URL, "http://www.mobile.ir/phones/prices.aspx?sort=date&dir=desc&brandid=0&terms=&duration=14&pagesize=50&price_from=-1&price_to=-1&provinceid=0&shopid=0&page=2");
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_URL, "http://www.php.net/");
curl_setopt($ch2, CURLOPT_HEADER, 0);

//create the multiple cURL handle
$mh = curl_multi_init();

//add the two handles
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);

$active = null;
//execute the handles
do {
    $mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) != -1) {
        do {
            $mrc = curl_multi_exec($mh, $active);
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);
    }
}

$dom = new DOMDocument('1.0', 'utf-8');
libxml_use_internal_errors(true);
$dom->loadHTML($mrc);
libxml_clear_errors();
$xpath = new DOMXpath($dom);

$data = array();
$table_rows = $xpath->query('//table[@id="price_table"]/tbody/tr'); // target the row (the browser rendered <tbody>, but actually it really doesnt have one)

if ($table_rows->length <= 0) { // exit if not found
    echo 'no table rows found';
    exit;
}
$s = "st\xECna";
foreach ($table_rows as $tr) { // foreach row
    $row = $tr->childNodes;
    if ($row->item(0)->tagName != '<tr class="carHeader"> <td>نام کارخانه </td> <td>نام خودرو </td> <td>قیمت نمایندگی (ریال) </td> <td>قیمت بازار (ریال) </td> </tr>') { // avoid headers
        $data[] = array(
            'Name' => trim($row->item(2)->nodeValue),
            'Price' => trim($row->item(4)->nodeValue),
        );
    }
}

echo '<pre>';

print_r($data);
Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.