Extracting data from a website

[unknown101]

 

Hi Guys,

 

Just wondering if theres any working sample code or tutorials around on extracting data from a given webpage URL.  I have searched google, but nothing particularly great has appeared.

 

Basically I want to search a bunch of given URL's for key words and if they are found return data to a text file, any help is appreciated..

 

Thanks in advance

[discomatt]

Easy enough... first question though, does your production server have fopen wrappers turned on?

[unknown101]

Yep it does.

[discomatt]

Here's a basic version of what you wanna do

 

<?php

// What do you wanna search for?
$search = 'PHP';

// Grab the source to a string
$source = file_get_contents('http://www.phpfreaks.com/forums/');

// Basic version
if ( stripos($source, $search) )
echo 'The website grabbed has '. $search .' in it';

echo '<br>';

// With a little more filtering
$count = substr_count( strtoupper($source), strtoupper($search) );
if ($count > 5)
echo 'The website grabbed has '. $search .' in it more than 5 times';

?>

[unknown101]

Hey, thanks for the code..Ill have a go with it shortly!

 

 

[unknown101]

Hey there ive tested the code, seems just what i need as a starting point.  But i now need to develop it slighly further.  Basically I want to read values from a file, then search for these and the results which have been found returned to another text file.

 

So am I correct in saying the process would be:

 

Open file, find first value say "music", assign this to the $search variable, run the script though, print result to a txt file if its been found i.e. The word "music" has been found" within the URL.  Then loop through this process reading in each value from the file until all have been searched? 

 

Thanks in advance.