Jump to content


web crawler/spider update notification

  • Please log in to reply
No replies to this topic

#1 warnockm

  • Members
  • PipPip
  • Member
  • 26 posts

Posted 14 September 2006 - 11:59 PM

Does anyone know of something that's been released that spiders websites?  Specifically, i want to give it a site (or a list of them) and have it spider them, staying only in that site, and download all of the documents, like .doc or .pdfs.  Then return periodically and report the changes and download the new ones.  Has anyone seen anything like this in PHP?  i'll code it myself if i have to , but might as well not reinvent the wheel.  Thanks!

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users