[SOLVED] preg_match_all Question

[JSHINER]

<?php
for ($n = 0; $n < 1000; $n = $n + 50)
{
   
$seed = "http://www.site.com/page=$n";
$data = file_get_contents($seed);

if (preg_match_all("/\http:[^\"\s']+/", $data, $links)) 

{

@header("Content-type: text/plain");

     for ($i=0;$i<count($links[0]);$i++) 
     {
        // echo $links[0][$i]. "\n";
          
       	echo substr($links[0][$i], 7, 100). "\n";

     }
}
}
?>

 

The above goes from page to page getting info. Works great.

 

However I need it to now dig deeper. I need it to take a preg_match_all for lets say example the text in a link that is name=TEXT&other=stuff - then go to a next page which has an address of http://www.site.com/page=TEXT (basically just insert "TEXT" and then gather info on all of those pages (TEXT1, TEXT 2, etc) - and display ALL results from those sub pages.

 

I hope this makes sense.

[effigy]

I don't follow. Are you saying you want to loop through $links to identify additional links that might have "page="?

 

If you're somewhat familiar with Perl, have you considered WWW::Mechanize?

[JSHINER]

Let's say I start at /page_a.html and on that page there are links: link.php?id=202 .... link.php?id=494 etc

 

And those all link to the same pages just different ID's

 

So my next step would be to gather info on all of the sub pages on that page so it would then go to link.php?id=$found

 

Does that make sense?

[darkfreaks]

http://us.php.net/manual/en/function.preg-match-all.php

[effigy]

Let's say I start at /page_a.html and on that page there are links: link.php?id=202 .... link.php?id=494 etc

 

And those all link to the same pages just different ID's

 

So my next step would be to gather info on all of the sub pages on that page so it would then go to link.php?id=$found

 

Does that make sense?

 

I think that's what I said. In this case you need to loop through your preg_match_all results and run a new expression against the results.

[JSHINER]

Yes.

 

However I am not familiar with Perl Is that the only way?

[effigy]

No, you can do this in PHP. I was just suggesting that if you were Perl savvy, that WWW::Mechanize is an awesome tool.

[JSHINER]

I tried with this:

 

<?php
$seed = "http://www.site.com/sub/index.htm";
$data = file_get_contents($seed);
if (preg_match_all("/http:\/\/www.site.com\/sub\/[^\"\s']+/", $data, $links)) 
{
     for ($i=0;$i<count($links[0]);$i++) 
     {
	$data_b = file_get_contents($links[0][$i]);

if (preg_match_all("/\http:[^\"\s']+/", $data_b, $links_b)) 
{

header("Content-type: text/plain");

     for ($i_b=0;$i_b<count($links_b[0]);$i_b++) 
     
     {
     
     echo $links_b[0][$i_b]. "\n";
     }
}
}
}
?>

 

But this does not work. Does anyone see the problem?

[effigy]

What kind of results are you getting? Can you reveal the site to provide us with test data?

[JSHINER]

Figured it out.

 

Thanks!