Akkari

Thanks a lot!

Hey there everyone, I'm using curl to retrieve webpages and lookup a certain string in their source code. Apparently a lot of websites use 303,304,...etc redirects for various reasons like: redirecting domain.com to www.domain.com redirecting domain.com to domain.com/portal ...etc Now the problem is that when curl hits any kind of redirect, it doesn't "follow" it but simply retrieves the page that has nothing it it except the header redirect and hence fails. Is there any way to make curl "follow" redirects and retrieve the page from where it's supposed to be? Thanks!

Thanks for the input everyone. I managed to solve the issue by changing the test website I was testing the script on. It turned out that the original test website http://site.com had a 3xx redirect to http://www.site.com which made curl fail to retrieve the website. I will open a new topic with that new issue, hopefully someone would be able to point me in the right direction. Thanks!

Thanks a lot for the response guys. @sumpygump The string is "/images/" (without the quotes of course) which occurs within URLs referencing the images folder of the website. So usually it'll occur as part of an image URL displayed on the page. @ManiacDan I think your suggestion might be useful in other situations so I'm looking it up now. However, in this particular situation I echoed out $content and it displayed the target site perfectly. Appreciate your responses, guys!

Hello there everyone, Been a while since I last posted. I've successfully retrieved external HTML pages using Curl, through the following code: $ch = curl_init("http://www.site.com/"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_BINARYTRANSFER, true); $content = curl_exec($ch); curl_close($ch); I then tried adding something, which I thought that there certainly has more to it than that but tried it anyway: if(strpos($content,"string_to_search_for") == false) echo "Not found."; else echo "Found."; Now this returned "Not found" every time for me, even if the string was present in that page source code. On a side note, I will be using this to evaluate hundreds, potentially thousands of websites to see if they have that string present in the source code. How long do you think execution time would be for say, a 1000 URL? And would there be a better approach to speed things up? Thanks!

SOLVED! A separate issue has arised so I'm gonna open a new thread. All I simply needed was to use the javascript's unescape(). By the way, I cannot find the mark as solved button. Is it something on my side?

Thanks for your eye opening response and sorry for the late reply. Actually when dumped in PHP it works. However, I think I should add a very important point. The arabic data is gathered through a form and processed through a $_GET. Meaning PHP receives the data from a URL. So all my concentration now is how to decode this: %u064A%u0627%u0633%u064A%u0646 which seemingly is how "arabic" looks in a URL. Thanks a lot.

Hey there everyone. Today isn't my lucky day since I'm dealing with international characters. (Arabic). What happens is the following: Arabic entered in form > saved to MySQL > Retrieved from MySQL > outputted as XML > used in an application (Google maps if it makes any difference). Plain and simple, the problem is that the final output is this: %u0645%u0639%u0644%u0645 which definetly isn't arabic. What I've tried so far: - Loading the Google Maps thing directly from XML which I manually created and put the arabic chars into -> WORKED! (meaning I cut out the saving and retrieval of info from the db to see where the problem may be coming from) - Set the database collation and charset to utf8_unicode_ci. (Also collation of the fields inside the table) -> NO LUCK! - Tried encoding all PHP files and html files involved in the process to UTF-8 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> -> NO LUCK! - Tried a suggestion from Google which said try to execute this query after connection with the db: mysql_query("SET NAMES 'utf8'"); mysql_query('SET CHARACTER SET utf8'); -> still no luck First time I get that frustrated. I guess the problem isn't with Gmaps and XML. Something to do with PHP & MySQL because when the raw XML files displays correctly, Gmaps works like a charm. And the only way I got the XML file to show correctly as said earlier was to manually create it and enter the arabic text in it. Thanks a lot for any help provided!

Hello there everyone. I'm pretty poor at Javascript but currently working on it. I have a simple question with regards to the current Google Maps javascript API. I'm trying to retrieve locations from MySQL and post them on a map as well as allow the user to enter a new location and store it in the db. So far I think I'm getting my hands on the right stuff. XML to output map: mysql > php > xml > javascript > display on map. In case of storing, javascript sending GET parameters to a PHP script which then parses and stores them in the db. The question is: With a lot of searching I think I found the tutorials that can help me achieve this. However, they are from 2007. This one: http://code.google.com/apis/maps/articles/phpsqlinfo.html and this one: http://code.google.com/apis/maps/articles/phpsqlajax.html These are the only tutorials I could find related to PHP and MySQL. Are they up to date or will they break when I try to implement them? I'm pretty confused with all the materials on code.google.com but found those to be the best suited to my needs. Are there things I'd have to consider that are no longer supported by the API or so? Can you recommend a better/ up to date tutorial? Thanks a lot.

hmmm understood thanks. So hidemyass doesn't use curl, what does it use then? Also in its case the IP I get when surfing through them is the IP of their server right? So how is it possible to allow users to use my server as a proxy with fully working javascript and stuff? I'd be grateful if you could point me to a tutorial or recommend a script. Thanks again and sorry for the headache!

That's a pretty helpful reply, thanks. I tried Curl (new to it) and was able to access from my PC and bypass protection. HOWEVER: 1. javascript did not seem to work. Can't those javascript issues be fixed on curl? 2. No links would work. (I think because they were relative links because when I clicked login it went to mydomain.com/login not twitter.com/login). Is there a way around that? How does a service like hidemyass.com manage to load sites flawlesly with all the js and everything? Thanks!

Yes you can say that. The government here in Egypt is censoring the web because of the protests. Perhaps you've already heard a bit. So I'm trying to code a simple solution that'll allow users to access twitter and bypass what's happening. They don't want people not talk. So there aren't any bad intentions as you were thinking Hope it's clear now!

Thanks for your response. Nope. I simply want to redirect them to the real site2.com but "anonymized". Just like how the network settings in firefox work as in the example above. Thanks!

Hello there everyone. I'll try to explain as thorough as I can so please bare with me a bit. One you want to surf the web through a proxy in firefox, you go to tools > options > network > settings and enter proxy details which for example are like this. 173.123.123.4 and port 8080. I want to do pretty much the same thing with php for my visitors. I have www.site1.com which will have all my scripts and stuff. I want it to somehow redirect or load to www.site2.com USING A PROXY so that when the visitor reach site2, it's as if they have edited their firefox settings to view site2 using a proxy. That way visitors will always be anonymous on www.site2.com. While searching, I found this: http://stackoverflow.com/questions/3889715/php-requests-through-proxy Which seems simple enough but does not work unfortunately. Not that I'm even sure that it's indeed what I want to do but it seems like it...lol. Thanks a lot for any help provided.

Finally decided to go with your first suggestion. Thanks.