Jump to content

Help with a SIMPLE Spider


JSHINER

Recommended Posts

<?php
$seed = 'http://www.site.com/?page=10';
$data = file_get_contents($seed);
if (preg_match_all("/http:\/\/[^\"\s']+/", $data, $links)) 
{

header("Content-type: text/plain");

     for ($i=0;$i<count($links[0]);$i++) 
     {
          echo $links[0][$i]. "\n";
     }
}
?>

 

The above gets all links from a page. But I need it to not only do page 10, but up to page 50. Can this spider multiple pages? If so, how can I do it?

 

 

 

Link to comment
https://forums.phpfreaks.com/topic/70964-help-with-a-simple-spider/
Share on other sites

for($i=10;$i<=50;$i++){
$seed = "http://www.site.com/?page=$i";
$data = file_get_contents($seed);
     if (preg_match_all("/http:\/\/[^\"\s']+/", $data, $links)){
     header("Content-type: text/plain");
     echo "Page $i\n:";
     for ($j=0;$j<count($links[0]);$j++){
          echo $links[0][$i]. "\n";
     }
     echo "\n\n\n";
     }
}

<?php
header("Content-type: text/plain");
for ($n=1;$n<50;$n++) 
{
$seed = "http://www.site.com/?page=$n";
$data = file_get_contents($seed);
if (preg_match_all("/http:\/\/[^\"\s']+/", $data, $links)) 
{
     for ($i=0;$i<count($links[0]);$i++) 
     {
          echo $links[0][$i]. "\n";
     }
}
}
?>

 

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.