Does anyone know of something that's been released that spiders websites? Specifically, i want to give it a site (or a list of them) and have it spider them, staying only in that site, and download all of the documents, like .doc or .pdfs. Then return periodically and report the changes and download the new ones. Has anyone seen anything like this in PHP? i'll code it myself if i have to , but might as well not reinvent the wheel. Thanks!
web crawler/spider update notification
No replies to this topic
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users