Jump to content

Archived

This topic is now archived and is closed to further replies.

clarencek

PHP Regexp file crawling challenge

Recommended Posts

Hi, I'm fairly new to php but am learning quite a bit.

I am trying to grab some information from a web page: http://en.wikipedia.org/wiki/Broadway_theatre.  What I want to get is just the table information by storing the theater, the show, the address, and when it opened.

Is it possible to do this via php and regular expression?

This is what I have so far:

<?php

error_reporting(0);

$url = 'http://en.wikipedia.org/wiki/Broadway_theatre';
$page = file_get_contents($url);
if (preg_match_all('/<td>.+?title=".+?">(.+?)<\/a>/im', $page, $links, PREG_SET_ORDER))

  for($i = 0; $i < count($links); $i++){
    print_r($links[$i]);
    echo "
";
  }


?>

The problem with it is that it picks up everything that starts with <td> includes title and ends in [/url], not just the theater name.  AS for how to pick up the address and opening date by itself, I have no idea how to separate out since they are enclosed in simple <td> </td> tags.  Ultimately I want to take these variables and stick it in the database.  I know how to stick it in the database.  The challenge is getting all the variables.

Thanks for any help,

Clarence

Share this post


Link to post
Share on other sites
If you only need to do this one time, an easier solution might be to highlight/copy the text from your web browser and paste it into your favorite macro enabled text editor to add the SQL necessary to insert it into the database.  ([url=http://www.crimsoneditor.com/]Crimson Editor[/url], for instance.)

Otherwise, this should work (untested):
[code]
/<tr><td><a.+?>(.+?)<\/a><\/td>\s+<td><i><a.+?>(.+?)<\/a><\/i><\/td>\s+<td>(.+?)<\/td><td><a.+?>(.+?)<\/a>,\s+<a.+?>(\d+)<\/a><\/td>\s+<\/tr>\s+/s
[/code]

[1] Theater
[2] Show
[3] Address
[4] Date
[5] Year

Share this post


Link to post
Share on other sites
Disregard earlier example.... this one [i]actually[/i] works. ::)
[code]preg_match_all('/<tr>\n<td><a.+?>(.+?)<\/a>.*?\n.*?<a.+?>(.+?)<\/a>.*?\n.*?<td>(.+?)<\/td>.*?\n.*?<a.+?>(.+?)<\/a>, <a.+?>(\d+)/i',$broadway,$matches);[/code]

Share this post


Link to post
Share on other sites
Hmm...I'm not sure I'm getting this to work.  I have put my code below with my comments and questions in the //Comments.

------------------------------
[code]error_reporting(E_ALL);

$url = 'http://en.wikipedia.org/wiki/Broadway_theatre';
$page = file_get_contents($url);[/code]

//This part below grabs just the table html, excluding the first <table> and the last </table> and stores it in $links[0][1]

[code]if (preg_match_all('/<table.class="wikitable">.+?<\/tr>(.+?)<\/table>/si', $page, $links, PREG_SET_ORDER));[/code]


// Then I put in your code

[code]if (preg_match_all('/<tr>\n<td><a.+?>(.+?)<\/a>.*?\n.*?<a.+?>(.+?)<\/a>.*?\n.*?<td>(.+?)<\/td>.*?\n.*?<a.+?>(.+?)<\/a>, <a.+?>(\d+)/i', $links[0][1], $matches));[/code]


// But when I print it out, I get a blank array.  Am I doing something wrong with your snippet?

[code]echo '<pre>' . print_r($matches,true) . '</pre>'[/code]

//Also, I was trying to keep date and year in one field although I don't know if that matters here.

Share this post


Link to post
Share on other sites
That preg_match_all() worked for me on that webpage when I tried it (although I had to save it locally -- file_get_contents() didn't work with that URL for some reason).  You don't need the first preg_match_all() function.  And don't worry about bringing out the date in one variable; just concatenate it when you put it in your database.

Does your program execution make it to the regex with the correct contents in those variables?  Really, it worked like a champ the other day.  :)

Share this post


Link to post
Share on other sites
Can you show me your complete code?

Here is what I have

[code]error_reporting(E_ALL);

$url = 'http://LocalCopyOftheHTMLPage';
$page = file_get_contents($url);

if (preg_match_all('/<tr>\n<td><a.+?>(.+?)<\/a>.*?\n.*?<a.+?>(.+?)<\/a>.*?\n.*?<td>(.+?)<\/td>.*?\n.*?<a.+?>(.+?)<\/a>, <a.+?>(\d+)/i', $page, $matches));

echo '<pre>' . print_r($matches,true) . '</pre>'[/code]

But the output I get on html page is such:

[code]Array
(
    [0] => Array
        (
        )

    [1] => Array
        (
        )

    [2] => Array
        (
        )

    [3] => Array
        (
        )

    [4] => Array
        (
        )

    [5] => Array
        (
        )

)[/code]


Results are blank.  Am I doing something wrong here?  I believe I'm using the exact code that you typed up.

Share this post


Link to post
Share on other sites
[code]<?php
$broadway = file_get_contents("Broadway_theatre.html");
preg_match_all('/<tr>\n<td><a.+?>(.+?)<\/a>.*?\n.*?<a.+?>(.+?)<\/a>.*?\n.*?<td>(.+?)<\/td>.*?\n.*?<a.+?>(.+?)<\/a>, <a.+?>(\d+)/i',$broadway,$matches);

echo '<pre>',print_r(array_slice($matches,1),TRUE),'</pre>';

?>[/code]

Share this post


Link to post
Share on other sites
Hmm... I don't get it.  I'm still getting the same blank array even after cutting and pasting your exact code.  I guess this works on your end?  Not sure why I'm not getting anything.

Share this post


Link to post
Share on other sites
Did you check your file contents variable to be sure it's full of the right thing?

Share this post


Link to post
Share on other sites

×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.