Jump to content

How To Scrape A Page That Uses Prompt Login


ultimatum

Recommended Posts

Hey guys,

 

I have a page that I need to scrape using PHP. This page requires login credentials (which I have) but I don't know how to pass those values to a login prompt (pop-up).

 

It's a simple prompt like this one: http://devot-ee.com/...ogin_prompt.png

 

Trying to just read the page using cURL or file_get_contents_page() function sends back "You are not authorized to see this page" without login questions or anything related.

 

If anyone knows of a solution please help me out.

 

Thank you,

 

 

- ultimatum

No, I'm not. If it was my site then I would just connect to the database. I'm parsing a website which I have a subscription to and parsing has been allowed to automate some steps in data analysis.

 

Are you talking about this CURLOPT_HTTPAUTH option?

 

If I get through the login, will curl pick up webpages with frames and will I be able to scrape information from them?

 

 

Thank you.

CURLOPT_USERPWD

 

Frames are dealt with by the browser - cURL will only give you the HTML that defines the frame, not the contents of the page referenced.

 

In general you do this kind of work by logging in (and making sure cURL is saving the cookies someplace you can reuse in subsequent calls) and hitting the exact URLs you need.

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.