Certain characters not showing up

[Janj]

Welp, I've encountered an error this time... Certain characters don't seemed to be displayed when I echo the data in PHP file. For example, the mdash (—) and the a with a tilde (which is necessary for certain places such as São Paulo). These show up as �. Is there any way to get them to show normally? I've tried surrounding them with `'s and putting \'s in front of them, but I can't seem to find out how to get them to work.

[Jessica]

You'll need to have your character encoding set to UTF-8, both in MySQL and your PHP generated HTML.

[Janj]

You'll need to have your character encoding set to UTF-8, both in MySQL and your PHP generated HTML.

Thanks!

[Janj]

Okay, I changed the collation to utf8_unicode_ci. This changes the encoding set to UTF-8, right? For some reason I'm still getting �'s. Is there another way to do it or something?

[Jessica]

If the table wasn't already UTF-8 when you inserted the data, I don't know that you can convert it back. 

[Janj]

I tried exporting and importing the table.  I also went in and redid a row, and it still shows up wrong.

[DavidAM]

You'll need to have your character encoding set to UTF-8, both in MySQL and your PHP generated HTML.

[Janj]

I know my HTML has to be UTF-8. Does that mean I need all my PHP in the body (since the HTML that makes the charset utf-8 is in the head)?

 

Thanks, I think this is the last question!

[Christian F.]

The functions you use in PHP to manipulate the data have to be UTF-8 compatible as well, yes.

Though this is only true for the functions that actually manipulate the string data that contains UTF-8 characters, it's better to treat all strings as if they were UTF-8. That way you are ensured that you will not encounter any future bugs, related to differing charsets.

 

That said, I suspect there are one or two places where you've might have forgotten to define the proper charset:

• On the database connection, with *_set_charset ().
  
• In the HTTP headers themselves, with the proper header () call.

 

The latter one is necessary because the meta-HTML headers are ignored by the browser, if they conflict with a HTTP header. So if your web-server is set to serve HTML as ISO-8859-1 by default (which a lot of them are), then your "meta http-equiv" HTML header is quite ineffective.

 

I also recommend reading the article "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)". As it says: It's the absolute minimum, and no excuses!

[Janj]

The functions you use in PHP to manipulate the data have to be UTF-8 compatible as well, yes.

Though this is only true for the functions that actually manipulate the string data that contains UTF-8 characters, it's better to treat all strings as if they were UTF-8. That way you are ensured that you will not encounter any future bugs, related to differing charsets.

 

That said, I suspect there are one or two places where you've might have forgotten to define the proper charset:

• On the database connection, with *_set_charset ().
  
• In the HTTP headers themselves, with the proper header () call.

 

The latter one is necessary because the meta-HTML headers are ignored by the browser, if they conflict with a HTTP header. So if your web-server is set to serve HTML as ISO-8859-1 by default (which a lot of them are), then your "meta http-equiv" HTML header is quite ineffective.

 

I also recommend reading the article "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)". As it says: It's the absolute minimum, and no excuses!

Thanks.  I tried this:

/* change character set to utf8 */
if (!mysqli_set_charset($con, "utf8")) {
    printf("Error loading character set utf8: %s\n", mysqli_error($con));
} else {
    printf("Current character set: %s\n", mysqli_character_set_name($con));
}

to test, and the output was "Error loading character set utf8:"  I don't understand.

[Christian F.]

Me using the * in the function name like that, and linking to the MySQLI version, was a subtle hint to tell you that:

1. I don't know what library you're using to communicate with the database.
   
2. And in any case, you really should be using MySQLI or PDO.

 

Since you got the error message, I'm assuming that you're still using the old and outdated mysql library. I recommend updating your scripts to use the current libraries, before the old one is completely removed from PHP. Alternatively, use the old mysql_* version of the function, in the meantime.

[Janj]

I found out that putting the HTML code for the character (such as  ) works without messing up.  Thanks for your help, I learned something!