psunshine

In what way do you mean lemon? I want to create an automated script thats run on site to acheive this. If you could give a bit more of a pointer or example I would be happy to hear.

Will the subjects in which they get the a's b's and c's also need storing or just how many of each grade?

It doesnt have to be curl, preferably whatever would be the best solution, im flexible. I want it to do the search for each keyword/keywords seperately and then the top x amount of results for each are displayed. So example input would be: football david beckham cricket hockey Results like this: football david beckham - en.wikipedia.org/wiki/David_Beckham , www.thefa.com/England/SeniorTeam cricket - en.wikipedia.org/wiki/Cricket , www.thecricket.nu hockey - www.nhl.com/ , en.wikipedia.org/wiki/Ice_hockey So you basically end up with a new line seperated batch google search with comma seperated results (or similar!).

Hi, Im pretty crap at php (more of a good googler and copy and paste guy) but this is something which I have used for scraping websites: $header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,"; $header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"; $header[] = "Cache-Control: max-age=0"; $header[] = "Connection: keep-alive"; $header[] = "Keep-Alive: 300"; $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7"; $header[] = "Accept-Language: en-us,en;q=0.5"; $header[] = "Pragma: "; $header[] = "Content-Type:text/html; charset=UTF-8"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'http://www.example.com'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)'); curl_setopt($ch, CURLOPT_HTTPHEADER, $header); curl_setopt($ch, CURLOPT_REFERER, 'http://www.google.com'); curl_setopt($ch, CURLOPT_ENCODING, 'gzip,deflate'); curl_setopt($ch, CURLOPT_AUTOREFERER, true); curl_setopt($ch, CURLOPT_TIMEOUT, 10); $output = curl_exec($ch); curl_close($ch); I then use regular expressions to filter out the crap and finally insert specifics into mysql tables. You could probably do similar but have two different columns (one for existing page content and one for most recently checked)which you could use in a compare function. This is probably a bad way of doing it and im sure some peeps on here can definitely give a proper solution

Any ideas??

Hi, This may seem like something simple to do for some but I for the life of me cannot get my head around it! Basically I want to be able to enter a list of keywords and retrieve the top 3 urls (from google for example) for each keyword seperately. Is this something that can be acheived with curl and php or something similar? If anyone has any ideas or direction it would be much appreciated. Thanks P

;D Thanks Rhodesa, worked like a charm and I kinda understand how it works as well which is a bonus of course So here is the full code for my specific data to array to mysql database. Hopefully it can help some other lost souls out there! <?php $lines = array( '<item><1>abc1</1><2>abc2</2><3>abc3</3><4>abc4</4></item>', '<item><1>abc1</1><3>abc3</3><5>abc5</5><6>abc6</6></item>', '<item><1>abc1</1><6>abc6</6><2>abc2</2><3>abc3</3></item>' ); $data = array(); $num = 1; foreach ($lines as $line) { preg_match_all('%<(\d+)>(.+?)</\1>%', $line, $matches); foreach ($matches[1] as $key => $value) { $data[$num][$value] = $matches[2][$key]; } ++$num; } foreach ($data as &$line) { for ($key = 1; $key <= 6; $key++) { if (!array_key_exists($key, $line)) { $line[$key] = 'none'; } } } $map = array( 1 => 'field1name', 2 => 'field2', 3 => 'field3', 4 => 'field4', 5 => 'field5', 6 => 'field6', ); foreach($data as $row){ $dbname = "databasename"; $dbuser = "meyou"; $dbpass = "abracadabra"; $dbserver = "localhost"; $tablename = "mytable"; mysql_connect($dbserver, $dbuser, $dbpass) or die('Error connecting to mysql'); mysql_select_db($dbname); $values = array(); foreach($map as $n => $key){ if(isset($row[$n])) $values[$key] = mysql_real_escape_string($row[$n]); } $sql = "INSERT INTO `brands` (`".implode("`,`",array_keys($values))."`) VALUES ('".implode("','",$values)."')"; mysql_query($sql) or die("Error: ".mysql_error()); } ?> I used regex preg_replace and preg_match/all on raw data to reach the initial $lines. Once again big thx. Pat

Yes, sorry, I think I was VERY unclear with what I said. Each array should be a new row and each value in array inserted to corresponding field. Any suggestions or pointers on how to accomplish the insert? Im a newbie and am trying to get my head around this but there arent many tutorials about on array to db table Thx Pat

Hi, I have looked using search and several other places trying to find a good detailed descriåtion on how to put differebt arrays in to my db table and have still not come up with anything concrete I have an array that looks like this: Array ( [1] => Array ( [1] => Description1 [4] => Name1 [6] => Date1 [8] => URL1 [2] => none [3] => none [5] => none [7] => none [9] => none [10] => none ) [2] => Array ( [1] => Description2 [4] => Name2 [6] =>Date2 [8] => URL2 [9] => none [10] => Picture2 [2] => none [3] => none [5] => none [7] => none ) [3] => Array ( [1] => Description3 [2] => Homepage3 [3] => City3 [4] => Name3 [6] => Date3 [7] => Year3 [8] => none [5] => none [9] => none [10] => Picture3 ) [4] => Array ( [1] => Description4 [2] => Homepage4 [3] => City4 [4] => Name4 [5] => Product4 [6] => none [7] => Year4 [8] => none [9] => none [10] => none ) ) ^Courtesy of some much appreciated help from effigy on this threadhttp://www.phpfreaks.com/forums/index.php/topic,204808.0.html I am searching high and low to find an answer or at least a good tutorial on how to insert this type of array in to my table where each value of each array represents a different column. Any ideas or pointers how to do this? Thanks Pat

WOW! You are an absolute god! It worked perfectly first time on over a thousand rows!!! So a thousand thank yous and thumbs up

Hi, Maybe this cant be done with regular expressions or maybe I am a complete newbie which is why I am posting here I have a list of items, each on newlines that can look like this: <item><1>abc123</1><2>abc123</2><3>abc123</3><4>abc123</4></item> <item><1>abc123</1><3>abc123</3><5>abc123</5><6>abc123</6></item> <item><1>abc123</1><6>abc123</6><2>abc123</2><3>abc123</3></item> I have used preg_match_all before to break each subpattern in the item in to an array which can then be inserted to database, but that has always been where the pattern in item follows the same structure for each line if (preg_match_all('/<item><1>(.+)</1><2>(.+)</2><3>(.+)</3><4>(.+)</4></item>/mi', $result, $matches)) where $result looks like this: <item><1>abc123</1><2>abc123</2><3>abc123</3><4>abc123</4></item> <item><1>abc123</1><2>abc123</2><3>abc123</3><4>abc123</4></item> <item><1>abc123</1><2>abc123</2><3>abc123</3><4>abc123</4></item> So how do I acheive the same when the pattern may or may not contain a certain pattern without missing any pattern? On the same note, is it possible to get $matches to return "none" for the lines which dont contain a pattern for a specific part of the item? For example I know that each item could have a possible 6 tags within but I dont which 6 it is and want to return "none" for the tags not included. I hope this makes sense and that someone can help. This has been driving me crazy trying to figure it out! Please let me know if it needs better explanation. Thanks Pat