PHP - Advanced fetch of external file?

[jmqbe]

Hi,

 

I am looking to fetch the contents of https://bb.kdg.be/webapps/blackboard/content/listContent.jsp?course_id=_98522_1&content_id=_1288143_1 in PHP

However, the page works with cookies, javascript redirect, and https. Therefor I cannot manage to make a local copy of the actual contents.

 

I have tried with curl, file_get_contents, file, fopen, and numerous user-written functions online, none of which worked.

 

Please make sure you TEST your answer before posting it, because it is probably not as obvious as it seems!

 

Thanks!

[teynon]

This code worked fine for me:

 

<?php
    $contents = file_get_contents("https://bb.kdg.be/webapps/blackboard/content/listContent.jsp?course_id=_98522_1&content_id=_1288143_1");
    echo $contents;
?>

 

 

I did have to enable the extension php_openssl first though.

[jmqbe]

Then somehow you also managed to get your PHP to save cookies, any idea how?

 

On 'my' server, the file_get_contents fetches a page that just contains a javascript redirect.

[teynon]

Heh, yep. Just ran a quick test so it was just redirecting to the contents.

 

Might I point out, that the reason this site is like this looks like it is purposely trying to prevent this type of thing. You could do it with another language. I don't know if you can do it with PHP. But it might be quite involved.

[jmqbe]

OK, thanks for trying.

I am aware that they do try to block this, but no harm is intended in my case

[xyph]

If they're trying to prevent this then they probably don't allow it.

[Philip]

I am aware that they do try to block this, but no harm is intended in my case

 

At least in your eyes it is no harm, but in the website owner's eyes, it is harm. The only reason you'd be doing this is to scrape their page, no?

[ManiacDan]

Please don't report posts simply because you don't like what they say.  If someone purposely obfuscates their website so you cannot make a copy of it, then they do so for a reason.  Downloading a copy of a website is illegal in the United States and in many other countries (I am not a lawyer). 

 

There exists a PHP class that emulates a browser {url removed}, complete with cookie storage and retrieval, but it does not parse javascript. 

[Pikachu2000]

Since you already know that the site blocks this, there's no reason to pursue this further. We don't provide help to people trying to circumvent systems that were put in place to prevent access to content in ways it wasn't intended to be accessed.

[ManiacDan]

Now that there's two mods saying the same thing, I'm closing this thread.