RSR Posted June 20, 2009 Share Posted June 20, 2009 I'm in the middle of a school project in a programming class and i ran into some problems. I'm using the following code to access another site and read the source code, which i use to post some other information on my own site. $url = "http://othersite.com/"; $fd = fopen($url, 'r'); while (!feof ($fd)){ $buffer = fgets($fd, 4096); $lines[] = $buffer; } fclose ($fd); This code worked for me, but then the other site started using a login popup thing (some .htaccess thing, I guess). I can login on the site by typing this in my browser: http://user:[email protected]/ So I thought that i could just add that to my code: $url = "user:pass@http://othersite.com/"; $fd = fopen($url, 'r'); while (!feof ($fd)){ $buffer = fgets($fd, 4096); $lines[] = $buffer; } fclose ($fd); This dosn't work for some reason? So my question is: How to read the source code of the site, when they are using a .htaccess login? Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/ Share on other sites More sharing options...
RSR Posted June 20, 2009 Author Share Posted June 20, 2009 I got this code from an example on php.net: $c = curl_init(); curl_setopt($c, CURLOPT_URL, "http://othersite.com/"); curl_setopt($c, CURLOPT_USERPWD, "user:pass"); curl_exec($c); curl_close($c); This code gets the source code of the site, but it dosn't use the username and password correctly, so I'm getting an error: "401 - Unauthorized: Access is denied due to invalid credentials." Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860135 Share on other sites More sharing options...
thebadbad Posted June 20, 2009 Share Posted June 20, 2009 You wrote user:pass@http://othersite.com/ instead of http://user:[email protected]/. I just tested the method on a folder protected with a .htaccess username and password, and it worked. Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860138 Share on other sites More sharing options...
RSR Posted June 20, 2009 Author Share Posted June 20, 2009 Oh, well I did type 'http://user:[email protected]' in the real code. This is the site I'm trying to access: htx.ots.dk/otgnet/adm/restr/portfolio/listeafleveringer.asp I get this error: Warning: fopen(http://[email protected]/otgnet/adm/restr/portfolio/listeafleveringer.asp) [function.fopen]: failed to open stream: HTTP request failed! HTTP/1.1 401 Unauthorized in mysite\....\funktioner.php on line 717 With this code: 716: $url = "http://user:[email protected]/otgnet/adm/restr/portfolio/listeafleveringer.asp"; 717: $fd = fopen($url, 'r'); 718: while (!feof ($fd)){ 719: $buffer = fgets($fd, 4096); 720: $lines[] = $buffer; 721: } 722: fclose ($fd); Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860144 Share on other sites More sharing options...
thebadbad Posted June 20, 2009 Share Posted June 20, 2009 Well, I don't know why it doesn't work. It did for me when testing on another site. Just double check your username and password. Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860150 Share on other sites More sharing options...
RSR Posted June 20, 2009 Author Share Posted June 20, 2009 Hmm, i just found out that it is only google chrome that accepts the url: http://user:[email protected]/otgnet/adm/restr/portfolio/listeafleveringer.asp Firefox pops up with the login form even though the username and password is in the url. IE gives me an error: Windows cannot find 'http://user:[email protected]/otgnet/adm/restr/portfolio/listeafleveringer.asp'. Check the spelling and try again. I'm starting to think that it might not be a .htaccess login? Could it be some asp thing? EDIT: A thing just crossed my mind. The username is '***\v-******' where the * are letters and numbers. When the url works in google chrome, I only type v-******: http://v-******:[email protected]/otgnet/adm/restr/portfolio/listeafleveringer.asp I have tried both ***\v-****** and v-****** in the php code, but neither of them worked. I'm thinking that it might be something with the '-' in the code that is causing some problem? Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860162 Share on other sites More sharing options...
thebadbad Posted June 20, 2009 Share Posted June 20, 2009 Oh, yeah, must be a Microsoft thing. But I've got no idea how to deal with that. Edit: If the username or password contains backslashes, you should enclose the whole URL in single quotes instead of double quotes to avoid any character translation. Try that and see if it does anything. Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860168 Share on other sites More sharing options...
RSR Posted June 20, 2009 Author Share Posted June 20, 2009 Hmm, I found out that the '***\' part isn't necessary. And the changing of " to ' didn't work. I guess it's not a .htaccess login then? Do you know any other way you could read the source code from a site? I tried some cURL example, this displayed the source code of the servers no-access page. So this couldn't login either. $url = 'htx.ots.dk/otgnet/adm/restr/portfolio/listeafleveringer.asp'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_USERPWD, "user:pass"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $output = curl_exec($ch); curl_close($ch); print $output; Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860173 Share on other sites More sharing options...
thebadbad Posted June 20, 2009 Share Posted June 20, 2009 must be a Microsoft thing. But I've got no idea how to deal with that. Edit: Found an article that explains the backslash thingy though: http://msdn.microsoft.com/en-us/library/72wdk8cc(vs.71).aspx But still don't know how to login via the URL. Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860183 Share on other sites More sharing options...
RSR Posted June 20, 2009 Author Share Posted June 20, 2009 Oh, you meant that the login was a microsoft thing and you don't know how to do that. Now I get it. But thanks for your time anyways (: Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860187 Share on other sites More sharing options...
RSR Posted June 20, 2009 Author Share Posted June 20, 2009 Edit: Found an article that explains the backslash thingy though: http://msdn.microsoft.com/en-us/library/72wdk8cc(vs.71).aspx But still don't know how to login via the URL. Oh, so the '***\' is just the domain. Hmm, it is strange that it is possible to login with or without the domain. Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860192 Share on other sites More sharing options...
mattal999 Posted June 20, 2009 Share Posted June 20, 2009 Well, to my knowledge, the http://user:[email protected]/ isn't a used thing anymore. It is more known for people to just type in their username and password on a login popup. Ok. Well since I don't have cURL installed on my local server (PHP4), I can't do any tests. Can you send me the username and password over a PM so I can test the script on my server? Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860354 Share on other sites More sharing options...
thebadbad Posted June 20, 2009 Share Posted June 20, 2009 Well, to my knowledge, the http://user:[email protected]/ isn't a used thing anymore. It is more known for people to just type in their username and password on a login popup. While I agree that protection of files via Apache's or maybe ASP.NET's authorization feature is rarely used these days (and accessing the files via the userinfo URI component even less), it is irrelevant to the case. He needs a way to automate the login via a script. @OP If you're studying on that school the site belongs to, I would ask an IT guy if it's possible to access the protected file via PHP somehow. Maybe FTP would be a possibility, if it's impossible the other way around. Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860382 Share on other sites More sharing options...
RSR Posted June 21, 2009 Author Share Posted June 21, 2009 Ok. Well since I don't have cURL installed on my local server (PHP4), I can't do any tests. Can you send me the username and password over a PM so I can test the script on my server? I'm afraid that I can't send you my username and password. But if you could send me your test ideas, I would be happy to test it myself. @OP If you're studying on that school the site belongs to, I would ask an IT guy if it's possible to access the protected file via PHP somehow. Maybe FTP would be a possibility, if it's impossible the other way around. Your right, I can ask my teacher if he knows any way to do it. But the problem is that the site is actually not owned by my school, but a larger school in another city. My school are only using it. Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860605 Share on other sites More sharing options...
thebadbad Posted June 21, 2009 Share Posted June 21, 2009 I just by chance stumbled upon an interesting cURL setting (CURLOPT_HTTPAUTH) that decides which authentication method to use with the connection. Have no idea why I didn't see it earlier. Try this, I actually think it should work: <?php $url = 'http://htx.ots.dk/otgnet/adm/restr/portfolio/listeafleveringer.asp'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY); curl_setopt($ch, CURLOPT_USERPWD, 'username:password'); curl_setopt($ch, CURLOPT_FORBID_REUSE, true); curl_setopt($ch, CURLOPT_FRESH_CONNECT, true); curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.0; da; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11'); $contents = curl_exec($ch); if ($contents === false) { trigger_error('Failed to execute cURL session: ' . curl_error($ch), E_USER_ERROR); } echo $contents; ?> Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860635 Share on other sites More sharing options...
RSR Posted June 21, 2009 Author Share Posted June 21, 2009 I just by chance stumbled upon an interesting cURL setting (CURLOPT_HTTPAUTH) that decides which authentication method to use with the connection. Have no idea why I didn't see it earlier. Try this, I actually think it should work: <?php $url = 'http://htx.ots.dk/otgnet/adm/restr/portfolio/listeafleveringer.asp'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY); curl_setopt($ch, CURLOPT_USERPWD, 'username:password'); curl_setopt($ch, CURLOPT_FORBID_REUSE, true); curl_setopt($ch, CURLOPT_FRESH_CONNECT, true); curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.0; da; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11'); $contents = curl_exec($ch); if ($contents === false) { trigger_error('Failed to execute cURL session: ' . curl_error($ch), E_USER_ERROR); } echo $contents; ?> It works! I actually have seen this htttpauth thing before, but I didn't know how to use it, so I forgot about it. Thank you alot for your time and help. I just love when some tricky little problem gets solved. Thank you (: Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860656 Share on other sites More sharing options...
thebadbad Posted June 21, 2009 Share Posted June 21, 2009 Glad it works, and you're welcome Link to comment https://forums.phpfreaks.com/topic/163015-fopen-with-a-htaccess-popup-authentication/#findComment-860676 Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.