Question About Webcrawler Using Php

[simpleorchid]

Hi,

 

I am trying to develop a webcrawler using php script. I am little confused about the method of webcrawler. In order to webcrawl, could i do that using a website that has select lists and checkboxes. if so, how?

 

The url will look like http://www.xxx.com/l...wPastNumbers.do and has list box and checkbox and a submit button.

 

How do I do it? Is there any tutorial related to this? Thanks

[BrettHartel]

I would suggest learning about php (www.w3schools.com) because developing a webcrawler takes a lot of knowledge. I don't think a newbie could undertake such a huge project.

[Christian F.]

Don't go to w3schools: They're outdated, and just plain wrong in cases. Worst of all: They don't care, as long as people continue to visit their site they're not going to change a thing.

 

What I recommend instead is the PHP manual, from which W3S has copied most of their material from anyway. You also need to look into cURL and DOMdocument, in particular.

There are plenty of tutorials online on how to do most things, unfortunately the quality is very varying. Most of the time the guides are completely devoid of any kind of security and/or completely outdated, and doesn't even notify you about it. That said, what you really should do before starting on your project is to get the PHP basics down, this includes everything up to and including the "Security" chapter in the PHP manual. Plus the cookies and sessions sections. At the very least.

Edited December 20, 2012 by Christian F.

[simpleorchid]

Thanks Christian..... I will take your advice. once again thank you.