austinthompson Posted February 4, 2012 Share Posted February 4, 2012 Ok, I am going to ask a question and try to explain my this so that it make sense. Wish me luck... Is it possible to write a script (or is there one out there already) that will search for specific results on multiple sites and then display the results in one location? Kind of the same way that dogpile does for search engines? I think that made sense. Feel free to ask questions for clarification if needed. Quote Link to comment Share on other sites More sharing options...
digibucc Posted February 4, 2012 Share Posted February 4, 2012 specific sites or all sites? anything is possible i would do it in 2 parts. a script that runs through all of your sites and searches them, and saves each result to a db. and then a script that reads the db and prints out all of the records matching the search term. Quote Link to comment Share on other sites More sharing options...
austinthompson Posted February 5, 2012 Author Share Posted February 5, 2012 Specific sites. I want to be able to go to two or three different sites and query them. Let's say I wanted to search two seperate dating sites and I was looking for redheads. I want to be able to display all my choices on a third seperate website. Quote Link to comment Share on other sites More sharing options...
thehippy Posted February 5, 2012 Share Posted February 5, 2012 Check out the websites in question to see if they have a web API to retrieve data from. For example if you were wanting to collate the results from Google, Yahoo and Bing, each of those websites have an API. Though, not all are free. The alternative, if the websites in question do not have an API is to retrieve the raw web document (HTML, XML) and search it for the data you want, called scraping. Be sure to read the website' terms of service, as they might have limitations on that type of activity (such as requests per minute), also have your scraper read and follow robots.txt After retrieving the data from the websites in question, its a simple matter of storage and display. Quote Link to comment Share on other sites More sharing options...
austinthompson Posted February 5, 2012 Author Share Posted February 5, 2012 The WTF emoticon does not begin to cover the look of confusion on my face. LOL These two sites have no API. I am probably biting off more than I can chew here. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.