html form data containing foreign languages; entering db

[Harry2o]

Hi,

 

here is the situation broken down to a simple scenario:

 

- an HTML page (partially created via inline PHP) shows data from a database and offers a form

- form data is POSTed to a PHP script

- PHP script will enter form data to MySQL database

 

I can't get this scenario to completely work with characters from foreign languages no matter which witchcrafty PHP functions about encodings I use.

 

First of all, I thought using UTF-8 was a good idea, so I tried to set everything to UTF-8 (charset via Content-Type as well as SQL collate).

 

The HTML page, visible to the user, will print the collected database data twice ( after it is collected by inline PHP):

- in a box visible as normal text

- in a form field textarea, so the user can edit the data (and still see the original, see above)

One might also want to note, that the data consists of "HTML text", so htmlentities() and html_entity_decode() need to be used at some point in time so that the user sees text instead of rendered HTML.

 

So, the user edits the textarea, clicks "Submit" and the "HTML text"-data is POSTed to the PHP script.

This PHP script just takes the data and does a simple UPDATE or INSERT.

 

Now imagine the user data (or data in DB) does contain foreign language characters (é, ñ, ä, ß and the likes):

Might someone be so kind and tell me, how I need to do this so the text box as well as the form field as well as the post-data (and also the database content) do show those characters instead of gibberish hieroglyphs?

 

Thanks

[JonnoTheDev]

After you connect to your database using mysql_connect() run the following query:

<?php mysql_query('SET NAMES UTF8'); ?>

 

At the top of your page add the following header:

<?php header('Content-Type: text/html; charset=UTF-8'); ?>

 

You should have no problems with those characters.

[Harry2o]

Thanks for your help.

 

1. should I do the SET NAMES mysql query every time I connect?

 

2. the page already holds

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

 

And to give you the full picture of the current state:

- the page shows the database content in textform (printed by inline PHP) via

nl2br( htmlentities( $row['code'] ) );

- the form data field (textarea) contains

htmlentities( $row['code'] );

 

- the php script gets the textarea data via

html_entity_decode( $_POST['code'] )

 

 

There's still no improvement ....

(except that if I do the SET NAMES in every connect, the foreign characters already in the database look different now, however still not correct)

 

 

 

[EDIT]

Correction:

Entering into database seems to work (when using SET NAMES UTF8).

However, when I want to print the data (which is HTML code) as text (NOT in the textarea), I use the above function, but it's still not the right characters. Do I need to use utf8_decode() or sth?

[JonnoTheDev]

1. should I do the SET NAMES mysql query every time I connect?

Yes, unless you have access to your my.conf file. You should really only connect to the database in one area i.e in a function or a common include, or a mysql db wrapper class i.e http://www.ricocheting.com/code/php/mysql-database-class-wrapper therefore it only takes one line throughout the whole application.

 

 

2. the page already holds

Code: <meta http-equiv="Content-Type" content="text/html; charset=utf-8">

Doesn't make a difference including the other php functions you are using.

 

[Harry2o]

Thanks for replying again.

 

Well, you got me wrong there .

I am already using common function for connecting to the database, so speaking about coding, it's no problem to add this.

My question was rather: I thought this would be a "once and forever" command, to set names to UTF8 and this be persistent to the database. However, honestly I don't really know what that command does.

 

 

And I think I fixed the issues now.

Key for me was to use htmlspecialchars() instead of htmlentities() when including the database content (which is HTML text) on the page without being rendered. It seems that htmlentities() would do sth weird with foreign characters whereas htmlspecialchars() leaves them alone.

 

Apart from that one use, I don't have to use any of those PHP functions now .

 

Thanks!

[JonnoTheDev]

Then your database must already be configured to use UTF8 as the default charset

[Harry2o]

Then your database must already be configured to use UTF8 as the default charset

However, without "SET NAMES UTF8" I see question marks on black squares  ...