oyoyg Posted April 28, 2006 Share Posted April 28, 2006 HiHow can I crawl a password-protected site? (I have a password, and its not illegal because its my own site) Quote Link to comment Share on other sites More sharing options...
ober Posted April 28, 2006 Share Posted April 28, 2006 You're going to have to explain yourself a little better. What are you trying to achieve? And if your crawler is smart enough to submit forms, you can certainly crawl whatever is past the authentication. Quote Link to comment Share on other sites More sharing options...
oyoyg Posted April 28, 2006 Author Share Posted April 28, 2006 [!--quoteo(post=369675:date=Apr 28 2006, 02:31 PM:name=ober)--][div class=\'quotetop\']QUOTE(ober @ Apr 28 2006, 02:31 PM) [snapback]369675[/snapback][/div][div class=\'quotemain\'][!--quotec--]You're going to have to explain yourself a little better. What are you trying to achieve? And if your crawler is smart enough to submit forms, you can certainly crawl whatever is past the authentication.[/quote]Yeah, I wan't to give some login information to my crawler, and it has to use it on a login form. After that the session should be set, and it should be possible to crawl the pass protected sites, right? Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.