thegladiator Posted April 1, 2012 Share Posted April 1, 2012 I am in the process of teaching myself PHP/MySQL. I have taught myself other programming languages in the past so its not that i can't learn. but as i'm new and trying to learn both at the same time, i'm needing some help with php / mysql to datamine either of the following sites. www.findchips.com www.eciaauthorized.com WHat i want to do is after putting in the part number, for each distributor that returns data i want to extract the following data for each distributor Quantity Break Resale at quantity break Current Stock I want to write the data into a field in the database that holds the p/n's and my company's resale. I'm wanting to do this as an exercise so that i can dynamically benchmark my resales against that of my competitors. To get a feel of the data that is returned, please use the following p/n. 1n4001-e3/54 any and all help would be greatly appreciated Quote Link to comment Share on other sites More sharing options...
salathe Posted April 1, 2012 Share Posted April 1, 2012 What exactly are you looking for help on? The concepts of scraping websites, or specifics for PHP, or…? How you would do this with PHP shouldn't be too dissimilar to how you would do it in other languages, have you done the same task in any other language before? Quote Link to comment Share on other sites More sharing options...
btherl Posted April 1, 2012 Share Posted April 1, 2012 Some people use preg_match() to extract data from HTML with regular expressions. You can also use a parser like this one: http://simplehtmldom.sourceforge.net/ Quote Link to comment Share on other sites More sharing options...
salathe Posted April 1, 2012 Share Posted April 1, 2012 Or much better tools for the job, like DOM. There are also tools to aid scraping, including ScraperWiki for doing the actual scraping and/or disseminating the retrieved data. Quote Link to comment Share on other sites More sharing options...
thegladiator Posted April 1, 2012 Author Share Posted April 1, 2012 Salathe, this is my first endeavor no matter the language for scraping. I said that I have taught myself other languages as i can pick up concepts quickly. What I am after help wise is the concept of scraping sites as it pertains to PHP. I'm assuming the logical thought process would be to attack the source code for the sites i've listed, however i have no idea where to actually begin. thanks What exactly are you looking for help on? The concepts of scraping websites, or specifics for PHP, or…? How you would do this with PHP shouldn't be too dissimilar to how you would do it in other languages, have you done the same task in any other language before? Quote Link to comment Share on other sites More sharing options...
thegladiator Posted April 2, 2012 Author Share Posted April 2, 2012 Gents, thanks for your collective helps. What I'm hoping i can derive is a solid base in implementing / understanding the DOM tools initially so that I no longer have to ask. The concept of screen scraping / data mining is one that I can visualize in theory, just now sure about how to begin making it work with code thanks coleman Salathe, this is my first endeavor no matter the language for scraping. I said that I have taught myself other languages as i can pick up concepts quickly. What I am after help wise is the concept of scraping sites as it pertains to PHP. I'm assuming the logical thought process would be to attack the source code for the sites i've listed, however i have no idea where to actually begin. thanks What exactly are you looking for help on? The concepts of scraping websites, or specifics for PHP, or…? How you would do this with PHP shouldn't be too dissimilar to how you would do it in other languages, have you done the same task in any other language before? Quote Link to comment Share on other sites More sharing options...
btherl Posted April 2, 2012 Share Posted April 2, 2012 The kind of google search term you need is "php dom html parser tutorial". One result is this, showing xpath queries: http://stackoverflow.com/questions/2571232/parse-html-with-phps-html-domdocument Here's another style: http://www.nicolaskuttler.com/post/php-innerhtml/ Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.