getting content from another website

[r-it]

alo there, i am requiring information and i don't think google understands what i am looking for. i am looking at integrating content from another website into mine but i don't know how to, i want to practice how this can be done for future because a lot of people are starting to require this. i will give you two scenarios:

1. say for example i have a sports site and i want to display the league standings for my country's soccer league, i could do it the old way of building a php application or manually editing the html or is there a way that i can do it (with the permission of the owner of the site i'm gonna be taking it from of course), and using my own stylesheet for the table?

2. i have a general news site and i want to feature the weather of the country on it, how would i go about doing this, could someone please help?

[Orio]

You could get the contents of a page using file_get_contents() or if you need more power (cookies/post data etc') you could use cURL to get the data.

From there it's all about string manipulation- preg_match()'s to fetch the data you need from the source code, and then all you need is to output it the way you want it.

 

Orio.

[thebadbad]

It's normally called web scraping/grabbing. In PHP I do it like this:

 

[*]Load external page source code into a variable with file_get_contents() or cURL.

[*]Filter out the content you want, using regular expressions (preg_match_all() normally) or simple string functions if it's less complex (strpos() and substr() e.g.).

[*]Use grabbed content in your context.

[r-it]

i get what you guys are saying, but then if the person on the other side decides to change their content then my site will no longer show the relevant info, because i'll be getting content from a certain spot if u know what i mean, on one of the bytes forums someone was talking about webservices but i have no idea where and how i would be able to implement them, and if the other party's data is not in xml what does that mean??

[thebadbad]

The idea behind web scraping is, that you'll always have the up to date content. Of course, if the content structure is changed on the page you're scraping, it will no longer work. Is that what you meant?

 

Web services would refer to web APIs, where a website is offering their content through e.g. a XML-file. It's always easier to use an API to get external content than scraping it, since that's what APIs are meant for.

[Orio]

Well if the site owner gives you permission to use his data, you could ask him to give you a few days heads up before he changes his site.

Another option is to simply ask from the site owner access to his raw data- if it's stored in a mysql db you could ask for a user that can retrieve data only, or if it's stored in some XML format you could ask access to that (this way you won't need to connect and grab the data in the ways suggested before by badbad and myself thus your scripts would run faster and more efficiently).

 

If you have permission to copy the data, I think you could ask for access to the data itself, avoiding the whole inconvenience of cases like you've suggested and more importantly- connecting and manipulating the data in the ways we suggested can be slow and a bit inefficient.

 

Orio.

[r-it]

hi again guys, this is relevant to this. I am trying to convert an ip address to a country using this tool: http://api.hostip.info/?ip=165.144.244.136 that i found here: http://www.houseoffusion.com/groups/cf-talk/thread.cfm/threadid:53430

 

the problem is i've never done something like this, i would like to send the ip address to the address using something and have the result returned to my site to save onto the database. I really have no idea where to start especially with xml with php, please help.