teamplaylotto Posted March 16, 2007 Share Posted March 16, 2007 I'm trying to fiddle with some code so I can output a form on an affiliate website of mine, the url is http://www.v-w-d.com/redir.asp?m=28917&target=/default.asp?goto=/vwdguest/email_results.asp I want to parse the page but only display the form so I can ask users to subscribe to results by email without having to look at the whole page but when I try and open the page with fopen or use curl, it comes back with "this page has moved, click here". It's probably to do with the site in question redirecting to another page but with a session or cookie, I have no idea! if someone could give me some code that will at least display the page the url corresponds to then I would be able to figure it out from there but at the moment, all I get is less hair as I pull it out! hope you can help Andy Link to comment https://forums.phpfreaks.com/topic/42963-trying-to-parse-a-webpage-but-server-sends-back-wrong-page/ Share on other sites More sharing options...
monk.e.boy Posted March 16, 2007 Share Posted March 16, 2007 Fire up Firefox, and install the Live HTTP Headers plugin. This shows you all the query response headers from your browser to the site, and back again. Taking a quick look at that pages shows various 302s (redirects) cURL will follow a 302 if you tell it: http://uk.php.net/manual/en/curl.constants.php But i may be better to NOT let is follow 302s, just examine the header info cURL passes you and look to see if it is 302, if it is dig the LOCATION field out and pass that into cURL again to follow the new URL. If you have Live HTTP Headers open it'll show you POST variables and COOKIEs being passed between you and the site. From that it should be easy to past this info into the cURL header field and mimic what your browser is doing. monk.e.boy Link to comment https://forums.phpfreaks.com/topic/42963-trying-to-parse-a-webpage-but-server-sends-back-wrong-page/#findComment-208702 Share on other sites More sharing options...
teamplaylotto Posted March 16, 2007 Author Share Posted March 16, 2007 thanks for that, I used the live http headers and saw that the page redirects to another page that accepts a cookie which shows the right page. now I need to figure out how to post the form fields to that page along with the right cookie info... thanks for your help monk.e.boy! Andy Link to comment https://forums.phpfreaks.com/topic/42963-trying-to-parse-a-webpage-but-server-sends-back-wrong-page/#findComment-208939 Share on other sites More sharing options...
teamplaylotto Posted March 16, 2007 Author Share Posted March 16, 2007 I did it!! I finally figured out curl with cookies and sessions! it took me all night but I started off small to see if I could grab a cookie and get the session id which was really helped by the live http headers plugin, I could see the exact page that was called from a form and what data was sent as POST and the cookie details. then I moved on to making a form, collecting a name and email address and then posting that to the results form along with a cookie that I collected earlier in the script. another big thanks monk.e.boy, this will really really help with my lottery promotion. I'll be able to capture users data for myself and then pass it on to the affiliate site to see if they are already subscribed. brill! Andy Link to comment https://forums.phpfreaks.com/topic/42963-trying-to-parse-a-webpage-but-server-sends-back-wrong-page/#findComment-209138 Share on other sites More sharing options...
monk.e.boy Posted March 19, 2007 Share Posted March 19, 2007 good work ;D There is a nice python bit of code called mechanize which does what you're doing. May be worth a look. monk.e.boy Link to comment https://forums.phpfreaks.com/topic/42963-trying-to-parse-a-webpage-but-server-sends-back-wrong-page/#findComment-210427 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.