Scrape website with curl

[stefan1982]

Hi!

 

I'm into a little project where I want to retrieve data from the swedish news website DN's SOS Live page (http://www.dn.se/nyheter/soslive).

On the page there is an iFrame and there is the data I want to retrieve. The iFrame address is: http://div.dn.se/dn/sos/soslive.php?id= ... er/soslive

 

Here is the code I have and it stoped working yesterday.

 

function curl_download($Url){ 
  
    // is cURL installed yet? 
    if (!function_exists('curl_init')){ 
        die('Sorry cURL is not installed!'); 
    } 
  
    // OK cool - then let's create a new cURL resource handle 
    $ch = curl_init(); 
  
    // Now set some options (most are optional) 
  
    // Set URL to download 
    curl_setopt($ch, CURLOPT_URL, $Url); 
  
    // Set a referer 
    curl_setopt($ch, CURLOPT_REFERER, "http://www.dn.se"); 
  
    // User agent 
    curl_setopt($ch, CURLOPT_USERAGENT, "MozillaXYZ/1.0"); 
  
    // Include header in result? (0 = yes, 1 = no) 
    curl_setopt($ch, CURLOPT_HEADER, 0); 
  
    // Should cURL return or print out the data? (true = return, false = print) 
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); 
  
    // Timeout in seconds 
    curl_setopt($ch, CURLOPT_TIMEOUT, 10); 
  
    // Download the given URL, and return output 
    $output = curl_exec($ch); 
  
    // Close the cURL resource, and free system resources 
    curl_close($ch); 
  
    return $output; 
} 

$sosURL = 'http://div.dn.se/dn/sos/soslive.php?id=p://www.dn.se/nyheter/soslive'; 
$data = curl_download($sosURL);

 

The data variable is empty.

 

I notice now that when I enter the webaddress "http://div.dn.se/dn/sos/soslive.php?id=p://www.dn.se/nyheter/soslive" there is no content, although this is the web address in the iFrame.

 

How has DN solved this and how could I get around it?

 

Best regards

Stefan