Non-standard character input parsing scripts?

[woolyg]

Hi all,

 

I'm facing a project that will take input of data from forms submitted by users, but in a lot of cases, the user text that is input will have non-standard characters - examples:

 

à (HTML &agrave or &#68)

ö (HTML &ouml or &#87)

- semicolons omitted the html code for ease of reading

 

Has anyone used any WYSIWYG textarea scripts, or general javascripts, that automatically changes non-standard character input by the user into its HTML equivalent, for ease of storage in a database and ease of display later on?

 

The last thing I want to have display on the site is those annoying little boxes when a browser can't understand a character....!

 

All input appreciated.

WoolyG

[Daniel0]

Just use the correct character set. If you do that you won't have to convert these to HTML entities.

[cags]

As Daniel0 says, providing the database and the pages are setup to use the correct character encoding it's not a problem. If however you do want a way to convert them anyway, it can be done using the htmlentities function.

 

htmlentities("àö", ENT_COMPAT, 'UTF-8');

[JonnoTheDev]

Use UTF-8 encoding.

When you connect to your database run the following query:

mysql_query("SET NAMES UTF8");

 

Set the content type header in your app:

<?php header('Content-type: text/html; charset=utf-8'); ?>

 

Shouldn't have any problems then

[woolyg]

Thanks everyone.

neil.johnson - should I run that mysql query *directly* after connecting to the DB in each case?

 

Also, if I changed the encoding of all pages in the project to utf-8 instead of ISO-8859-1, should that suffice? I'm reading about the different types of encoding now.

 

Cheers for the valuable info - really appreciated.

Wooly

[cags]

Generally speaking I would have something along the lines of...

 

mysql_connect('blah', 'blah', 'blah');
mysql_select_db('blah');
mysql_query("SET NAMES UTF8");
mysql_query("SET CHARACTER SET UTF8");

Yes, that should suffice with regards to the encoding of the pages.

[waynewex]

Doesn't changing DB encoding mid-connection create a security hazard?

[cags]

You aren't changing the encoding of the actual db you are changing the encoding type of the connection to the db.

[woolyg]

I've been reading a bit about changing tables from latin1 ut utf8. Anybody got any opinions on this also?

 

Thanks for input so far.

Wooly

[cags]

Yes, that is another step you will probably have to take to get it working correctly.

[Daniel0]

Latin-1 can display à and ö just fine. Strictly speaking there is no need for UTF-8 if you restrict yourself to Eastern European languages. The only requirement is that your character set of choice has to be consistent throughout the entire system.

[woolyg]

Cool. Thanks guys for the informative replies. The project will start in english, and eventually incorporate indian characters, as well as some eastern european languages as well, so I'll take your advice, do some intense reading on the whole process, and get to changing my settings around.

 

Waynewex - as a fellow Irishman I am beside you in mourning the death of decency, honour, and innocence of the beautiful game. Tiocfaidh ár (fraincis) lá

 

Cheers everyone

[JonnoTheDev]

as a fellow Irishman I am beside you in mourning the death of decency, honour, and innocence of the beautiful game

vava voom. what a cheating ba***rd