s0c0 Posted March 29, 2007 Share Posted March 29, 2007 Can anyone point me to a how to on building a web spider or at least something that covers the concepts? Quote Link to comment Share on other sites More sharing options...
Lumio Posted March 29, 2007 Share Posted March 29, 2007 You're in the wrong board. And what is a web spider? Quote Link to comment Share on other sites More sharing options...
s0c0 Posted March 29, 2007 Author Share Posted March 29, 2007 You're right, this is in the wrong forum I had meant to post this in the help section. My apologies I was tired when I posted this last night. Mod's, can you please move my post? To answer your question, a web spider is a program that crawls the web and pulls back information. All search engines (google, yahoo, msn) use a spider to crawl the web looking for new sites to index. I need to bring my self up to speed on several aspects of PHP for a job I will be taking soon and they have asked that I develop a basic spider. Quote Link to comment Share on other sites More sharing options...
Lytheum Posted March 30, 2007 Share Posted March 30, 2007 From what I've read, Perl would be a better language to make a spider. But as I can see - you're dealing with PHP. I also read that this function helps a lot when you try to tackle something like this in php. http://sourceforge.net/projects/snoopy/ Perhaps a start would be to read up on php socket functions/xml parsing. I'm not sure how one would go about something like this Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.