Getting info from clients for portal

[Bleem]

Hello, 

 

I'm trying to create a portal that will display information from other websites (similar to travelsupermarket.com, lastminute.com, etc) but I was wondering what are the best ways/different options for getting client data into our database?

 

The methods I've come up with so far are:

1. Admin section

2. Copying a specifically generated file from the client and using a php file on a cron job to

    parse and sync the data

3. Processing an RSS feed

4. Webscraping - manually configured for each site

5. Them FTPing a file onto our server which we later process

 

 

Kind of things I was wondering about:

 

* Being able to query their database directly?

* Best methods of generating the actual data files (should we prepare easily configurable scripts

    in php/asp/etc which they then host on their server?)

* Any other methods we might be able to use

 

 

Any help would be greatly appreciated!

 

 

 

 

[Buyocat]

Well the first thing that comes to mind is Don't!  However if you're hell bent on making a site which relies on other sites' data I think the first step would be to start talking with them.  They may be willing to publish an API that you can call, or something similar.  It may cost you some money, but in the long run it will make your life much easier.  The alternatives, such as screen scrapping, all rely on the client's data and structure not changing, which I think is a bad assumption.  Also, I at least would not want people harvesting my data and using it in some way I didn't intend, so I think it's just good manners to swing your idea past them.

 

If they are open to the idea of sharing their data with you then a web service is really the best way.  In this case you would call some server and pass some XML and retrieve a response in the form of more XML.  Since it takes time and money to develop a web service they may not be willing to do that, so in that event I would look for a way to have them export their database entries and either upload them through an admin or web service you've published.