Jump to content

etrader

Members
  • Posts

    315
  • Joined

  • Last visited

    Never

Posts posted by etrader

  1. It is easy to get image or link by DomDocument, but I did not find a way to get image with its target link. Imagine a html as

    <div class=image>
    <a href='http://site.com'><img src='imagelink.jpg'></a>
    </div>

    How to get both the image link and href?

    $dom = new DOMDocument();
    @$dom->loadHTML($html);
    $xpath = new DOMXPath($dom);
    $hrefs = $xpath->evaluate("/html/body//div[@class='image']");
    for ($i = 0; $i < $hrefs->length; $i++) {
    $href = $hrefs->item($i);
    

    Now to get the image and its href, we need first getElementsByTagName('a') and getElementsByTagName('img') but they do not work inside foreach.

     

    What's your idea?

  2. Thanks for your informative reply. One more question. Does the order has an influence on cURL processing? I mean changing the order of lines in

    curl_setopt($ch, CURLOPT_USERAGENT, 'Googlebot/2.1 (http://www.googlebot.com/bot.html)');
    curl_setopt($ch, CURLOPT_URL,"http://www.site.com");
    curl_setopt($ch, CURLOPT_FAILONERROR, true);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_REFERER, 'http://www.google.com/bot.html'); 
    curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
    curl_setopt($ch, CURLOPT_TIMEOUT, 10);
    

  3. When making a fake identity in cURL, we use

    curl_setopt($ch, CURLOPT_USERAGENT, 'Googlebot/2.1 (http://www.googlebot.com/bot.html)'); 

    but which is better for the referrer:

    curl_setopt($ch, CURLOPT_AUTOREFERER, true); 

    or

    curl_setopt ($ch, CURLOPT_REFERER, "http://www.google.com/bot.html");  

    :shy:

  4. Imagine an html with the following structure

    <div class="item"> 
        <div class="title"> 
            <a class="title" href="http://www.domain.com/title.html">Title is here</a>   
        </div> 
        <div class="image"> 
            <a href="http://www.domain.com/title.html"><img src=image.jpg /></a>
        </div>
    </div>
    

     

    How to make an array containing $title - $url - $image_url ?

  5. download the file first?

     

    (untested)

    $cachepage = "cache/pagename.html";
    $external = "http://google.com";
    if(!file_exists($cachepage)) {
    	$ch = curl_init($external);
    	$fp = fopen($cachepage,'w');
    	curl_setopt($ch, CURLOPT_HEADER, false);
    	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    	curl_setopt($ch, CURLOPT_FILE, $fp);
    	curl_exec($ch);
    	curl_close($ch);
    	fclose($fp);
    }
    $html = file_get_html($cachepage);
    unset($cachepage);
    

     

    The only problem I have is that it writes pagename.html with chmod 644, and then unset cannot delete it or re-writing in the next run.

  6. I successfully load a page by simple_html_dom.php (developed in simplehtmldom.sourceforge.net) as

    $html = file_get_html('externalpage');

     

    But sometimes this make a high load on CPU and the page does not load for a long time (probably due to the external site server). How can I skip the process when it is not normal to avoid high CPU usage?

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.