shlomikalfa Posted March 28, 2008 Share Posted March 28, 2008 hey there, i'm in a desperate need of understanding some issues with UNICODE, i'll start by describing what is it that i need to do. I need to form a TCP-IP POST operation into a PHP page which receives it and saves it into a MySQL data-base. A. the required info is grabbed, converted to a proper HTML ENTITY, then is encrypted into a valid TCP-IP send and sent to the PHP page. [DONE OK.] B. the PHP page now decrypts it and receives the HTML ENTITY. [DONE OK.] C. the HTML ENTITY is to be saved in a MySQL server as the normal string/ UNICODE. [FAILS HERE!!!] as you can see i have issues with conveying the HTML ENTITY into the MySQL database, when trying to use "html_entity_decode" before sending the content to the MySQL database it seems like it fails and returns the HTML ENTITY as is with no changes... How do i Convert HTML ENTITY into a proper UNICODE string ?! -->> i want to save the content in the database as it is received in the first place before any changes... so that the database will still be searchable !!! THANKS IN ADVANCE! Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 is there actually no way to save UNICODE strings ie. שלום into MySQL as is ?! cause i want to perform searches using that same strings later on.... Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 Yes you can, you just need to tell the database that it's utf-8 you're saving (and the database collations/charsets settings should be set to utf-8 of course): Use this before your queries: <?php mysql_query("SET NAMES 'utf8'", $connection); mysql_query('SET CHARACTER SET utf8', $connection); ?> Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 oky that solves one part of the issue... The שלום UNICODE string is now save'able on the database. However, How can i transform the HTML ENTITY [ie. מי] to the normal UNICODE string in PHP? seems like using: "html_entity_decode" function doesn't give a UNICODE string but an HTML ENTITY.... Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 You're right that <?php $str = 'ampersand#1502;'; echo html_entity_decode($str); ?> outputs a HTML entity (because the character isn't found in ASCII I guess), but I found this to work: <?php $str = 'ampersand#1502;'; echo html_entity_decode($str, ENT_COMPAT, 'UTF-8'); ?> EDIT: ARGH! The ampersands are converted by the forum, so please change 'ampersand' with '&' above. Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 welll bah!!! my "Title" table entry is set for "utf8_unicode_ci" collation and after using the following code with: שלומיק i get the same thing [שלומיק] in the MySQL database - WHY ?! mysql_query("SET NAMES 'utf8'", $link); mysql_query('SET CHARACTER SET utf8', $link); ... $preTitle = mysql_real_escape_string($_POST['pTitle']); $Title = html_entity_decode($preTitle, ENT_COMPAT, "UTF-8"); echo Title; ... /// Inserting to table. all i want to do is to get that info sent to the PHP page: שלומיק to show on the MySQL table as: שלומיק [the decoded unicode string it represents] HOW DO I ACHIEVE THAT ?! PLEASE HELP, ANY HELP WILL BE APPRECIATED !!! Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 It is working for me :-\ <?php $str = 'שלומיק'; echo html_entity_decode($str, ENT_NOQUOTES, 'UTF-8'); ?> outputs שלומיק while <?php $str = 'שלומיק'; echo html_entity_decode($str, ENT_NOQUOTES); ?> outputs שלומיק Also, when you're already escaping the string, I would use ENT_NOQUOTES as the second parameter in the html_entity_decode function, to leave already escaped single and double quotes. But that has nothing to do with the entities. Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 Oh! I think I found your problem. I just noticed that "Support for multi-byte character sets was added" in PHP 5.0.0. Would you happen to use an earlier version? I've tested the working script above on PHP 5.2.2 and PHP 4.4.7, and the decode fails in the latter. Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 well since it didn't work for me online, [as it does work OK localy !!!] i'm using a different way to transfer the data into the PHP page... Now, i can get the UNICODE string [שלומיק] to appear in the PHP page correctly [after the send and the conversions] but after i'm using the following to send it to the MySQL database, i get it in there as: [ùìåîé÷] which is not quite the same $sql = 'INSERT INTO Table ( `Var1`, ...) VALUES ("'.$Title.'", ...) mysql_query("SET NAMES 'utf8'", $link); mysql_query('SET CHARACTER SET utf8', $link); $result = mysql_query($sql); :'( :'( :'( why is there something that always messes things up ?! Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 Remember to set the charset for the page you're displaying these things in to UTF-8, if you wanna keep things as easy as possible. <meta http-equiv="content-type" content="text/html; charset=utf-8" /> Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 well... i don't get what u mean sir. when i open "phpmyadmin" and click the Edit button then i write something in hebrew inside an entry ie, שלומיק and then i click save - it works perfectly... however, when Insert it through the PHP page it shows as i've mentioned.... Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 Well, did you read my post concerning the version of PHP? Seems like you just skipped that. Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 of course i have it's just that i wasn't that sure about using PHP5... :'( i never thought it'll be that messy !!!! [i hate PHP!] the thing is all i want to do is just the same as it is in PHPBB3 that you can place as a post subj whatever u wish, ie. "ффлыдоврфы ששחלדף does this work ?!" just tried that as a subject and it works just fine... and when u enter the table through phpmyadmin you can still find it just the same "ффлыдоврфы ששחלדף does this work ?!" i've just tried: "aasd שלומי ффыввогйн" it was Inserted and all should be oky, however, when i try and look at the entry through phpmyadmin -> "aasd שלומי ?????????" is what i saw :'( --> i've just checked and it seems like PHPBB3 is using "PHP Version 4.4.8" so i don't think this is the only reason as to why has my script failed the process. Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 But I'm pretty sure it's the reason that html_entity_decode fails on you. I would consider to stick with UTF-8 throughout the whole process. Why wouldn't you? Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 well.. sir, after your last comment i've went back to UTF8... It seems like you were right, the: html_entity_decode($Title, ENT_NOQUOTES, "UTF-8"); failed because i was using the wrong PHP version. [which is now changed to PHP5] However the other issue of seeing the data on the other side [phpmyadmin] is still there... -> i send: "akunhe שלומיק флгтру" -> which is then converted [before send] to: "akunhe שלומיק флгтру" -> Then once received by the PHP page it is being converted by the above line to: "akunhe ׳©׳œ׳•׳ž׳™׳§ ׁ„׀»׀³ׁ‚ׁ€ׁƒ" -> which is then sent to the MySQL db by: $sql = 'INSERT INTO Table ( `Var1`, ...) VALUES ("'.$Title.'", ...) mysql_query("SET NAMES utf8"); mysql_query('SET CHARACTER SET utf8'); $result = mysql_query($sql); ] -> when i look it up in phpmyadmin i get that: "akunhe שלומיק флгтру" it's not yet what i'm trying to achieve :'( Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 Edit: Disregard what was here.. All I can say is, that the code you're using is working for me. When you view these characters in your browser, are you sure that you've set the charset to UTF-8 with a meta tag (between <head> and </head> in you HTML)? Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 i'm not viewing these through a web-page of mine, i'm viewing it through phpmyadmin and as i was saying when i'm opening the PHPBB3 tables and look for the entery: "ффлыдоврфы ששחלדף does this work ?!" [for example.] it shows it just fine. also if i use the edit button and write something in heb/russian and then click the go [save] button it shows the same... but with the data that has been INSERTED through the PHP page it shows as i showed before: "akunhe שלומיק флгтру" please if you have any idea for me to try... please help me... Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 It must be when you convert the entities something goes wrong. Have you tried the scripts I've posted standalone? <?php $str = 'שלומיק'; echo html_entity_decode($str, ENT_NOQUOTES, 'UTF-8'); ?> outputs (when browser is set to UTF-: שלומיק What's your output? Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 29, 2008 Author Share Posted March 29, 2008 ׳©׳œ׳•׳ž׳™׳§ my output for the same... oky i get what u'r saying.... when i change the pages' encoding to UTF8 it appears oky... but how come it's not the same in phpmyadmin ?! changing the pages' encoding there isn't that helpful... it stays as i told before... Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 29, 2008 Share Posted March 29, 2008 I'm pretty sure your browser is not set to UTF-8, then, 'cause when I change the character encoding to ISO-8859-1, I get similar chars compared to yours. If you're using Firefox, you can change the charset to UTF-8 under View > Character Encoding > Unicode (UTF-. (If you haven't already set it using the meta tag). Going to bed now, I will check back tomorrow Edit: Maybe an idea: Try an utf8_encode() on the string before inserting to the database. Might solve it! Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 30, 2008 Author Share Posted March 30, 2008 well.. -> i'm 100% using UNICODE(UTF- in my browser. -> i've tried using it with/without that "utf8_encode()" I'm freakin crying here soon... why can't it just freakin' work... LET ME BE damn u PHP !!!! by the way, how does this function work: "unicode_decode()" cause it seems to not function at all, however it does exists in the PHP manuals... Quote Link to comment Share on other sites More sharing options...
shlomikalfa Posted March 30, 2008 Author Share Posted March 30, 2008 well... i've tried what you've said before about outputting the details into a web-page... it seems to be kept in the db just fine... just that stupid phpmyadmin site is showing it with mistakes.... so THANKS A LOT !!!! Quote Link to comment Share on other sites More sharing options...
thebadbad Posted March 30, 2008 Share Posted March 30, 2008 Glad it seems solved then! Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.