Extract data from web page

[zavin]

Is it possible to write a php script that would extract data from an external web page. I've been working with php for a few years as a hobby, but I've never seen this done or needed a reason to until now. Thanks in advance for any responses.

[QuickOldCar]

Use curl

 

http://php.net/manual/en/book.curl.php

[johnny86]

Depends on what you need to extract from there.. Do you want something from the source code? Or transfer something like a file?

[zavin]

QuickOldCar - Thanks I'm going to start studying up on cURL to see if it's what I need.

 

johnny86 - I'm wanting to extract information. For example if a web page had a list of prices and I wanted my page to list some of those prices based on the item from that page I want to list.

 

I hope that makes since.

[QuickOldCar]

May want to use http://simplehtmldom.sourceforge.net/ but also curl to resolve the urls first.

 

If look into dom I feel that alongside curl can do what you need to.

 

Or just everything in curl, up to you.

[johnny86]

Or you can just fetch the source code of the site with $source = file_get_contents('http://site.to.fetch.from/');

 

Then you could use preg_match to extract what you need. Which would be bit faster than using simplehtmldom. But you can also use simplehtmldom with your $source..

[zavin]

I think $source = file_get_contents('http://site.to.fetch.from/'); will work for me, but I'm also going to look into simplehtmldom a little more. The first problem I have is that the site I'm trying to get info from requires a log in and of course when I try to log in it looks for the log in page for the site on my server. Is there a work around for this?

[lastkarrde]

of course when I try to log in it looks for the log in page for the site on my server

 

I don't understand.

 

cURL can store cookies. Do a request to the login page, passing the appropriate login parameters via POST. Then do another request to the page you want to access.