Special characters work only on some servers only. (flat file cms)

r0b · June 2, 2011

Someone recently reported that a simple 10kb CMS I made doesn't work with special characters on some servers.

I'm not sure why this happens and I'm looking for some help. The CMS files are attached and I'm also pasting them over here, since there are only 2 main files.

On some servers, when entering a special character, it returns false. And on some it saves the character, but once it reads it again, it becomes a questionmark. You can have a test on how it doesn't work over here. (password for login is admin)(click anywhere on the content to edit it).

Special characters to test: š č ž. (or any other special characters like the german umlauts: ü ö)

This is the index.php

<?php

function getSlug( $page ) {
   $page = strip_tags( $page );
   preg_match_all( "/([a-z0-9A-Z-_]+)/", $page, $matches );
   $matches = array_map( "ucfirst", $matches[0] );
   $slug = implode( "-", $matches );
   return $slug;
}
   $page = $_REQUEST['page'];
   if(!$page) $page = "Home";

   $contentfile = $page = getSlug( $page );
   $content[0] = @file_get_contents("files/$contentfile.txt");
   if(!$content[0]) $content[0] = "Your page named <b>$page</b> is created.<br /><br />\n\nClick here to start editing your newly created page!";
   
   $title = @file_get_contents('files/title.txt');
   if(!$title) $title = "Your title here";
   $slogan = @file_get_contents('files/slogan.txt');
   if(!$slogan) $slogan = "- Your slogan over here!";

   $menu = @file_get_contents('files/menu.txt');
   if(!$menu) $menu = "Home";
   
   $description = @file_get_contents('files/description.txt');
   if(!$description) $description = "Enter your website description over here!";
   $keywords = @file_get_contents('files/keywords.txt');
   if(!$keywords) $keywords = "Enter, your, keywords, for, your, website, over, here";

   $copyright = @file_get_contents('files/copyright.txt');
   if(!$copyright) $copyright = "Your website (c) 2011";
   $mess = 'Powered by <a href="http://krneky.com/en/wondercms">WonderCMS</a>';

//config section
$hostname = $_SERVER['PHP_SELF'];
$hostname = str_replace('index.php', '', $hostname);
$hostname = str_replace($page, '', $hostname);

   $theme = $_REQUEST['theme'];
   if( !file_exists("$theme.php") ) $theme = "default";

   $cookie      = 'wondercms';	// cookie ame
   $expirytime  = time()+86400; // expire time

   if(isset($_REQUEST['logout']))
   {
   setcookie($cookie,'',time() - 84000); // remove cookie/
   header('Location: ./');
   exit;
   }
   $password = @file_get_contents("files/password");
   if(!$password)
   {
   savePassword("admin");
   }
   if($_COOKIE[$cookie])
   {
   $lstatus = "<a href='$hostname?logout'>Logout</a>";
   }
   else	$lstatus = "<a href='$hostname?login'>Login</a>";
   
   if(isset($_REQUEST['login']))
   {
   getLoginForm();
   }
   require("$theme.php");

// Functions
   function editTags()
   {
   global $cookie;
   if(!$_COOKIE[$cookie]) return;

   echo  "<script type='text/javascript' src='./js/editInplace.js'></script>";
   }

   function displayMainContent()
   {
   global $cookie, $content, $page;
   
   if($_COOKIE[$cookie])
   {
	   echo "<div class='title'><div id='change'><span id='$page' class='editText'>$content[0]</span></div></div>";
   }
   else
   {
	   echo $content[0];
   }
   }

// display section content
   function displaySectionContent($cnum)
   {
   global $cookie, $content;
   
   if($_COOKIE[$cookie])
   {
	   echo "<div id='change'><span id='$cnum' class='editText'>$content[$cnum]</span></div>";
   }
           else	echo $content[$cnum];
   }

   function displayMenu($stags,$etags)
   {
   global $menu;
   
   $mlist = explode("<br />",$menu);
   $num = count($mlist);

   for($ix=0;$ix<$num;$ix++)
   {
	   $page = trim($mlist[$ix]);
	   if(!$page) continue;
	   echo "$stags href='$page'>$page $etags \n";
   }
   }
   
   function getLoginForm()
   {
   global $content, $msg;
   
   $msg = "";
   
   if (isset($_POST['sub'])) loginSubmitted();
   $content[0] = "
<center>
<form action='' method='POST'>
<h2>Password</h2>
<input type='password' name='password' /><br />
<input type='submit' name='login' value='Login'>
<h2>$msg</h2><br />

<script src='js/editInplace.js'></script> 
<div class='all'>
<a href='javascript:showhide()'>Click to change your password</a> <br /><br />
<div id=hide style='display: none;'>
Type your <b>old</b> password above, and your new one in the field below.
<h2>New Password</h2>
<input type='password' name='new' /><br />
<input type='submit' name='login' value='Change'>
<input type='hidden' name='sub' value='sub'>
</div></div>
</form></center>";
   }
   
   function loginSubmitted()
   {
   global $cookie, $password, $msg, $expirytime, $submitted_pass;
   
   $submitted_pass = md5($_POST['password']);

   if ($submitted_pass<>$password)
   {
	   $msg = "<b class='wrong'>Wrong Password</b>";
	   return;
   }
   if($_POST['new'])
   {
	   savePassword($_POST['new']);
	   $msg = "Password changed!<br /><br />Please login again.";
	   return;
   }
   setcookie($cookie,$password,$expirytime);
   header('Location: ./');
   exit;
   }
   
   function savePassword($password)
   {
   $password = md5($password);
   $file = @fopen("files/password", "w");
   if(!$file)
   {
	   echo "<h2 style='color:red'>*Error* - unable to access password</h2><h3>But don't panic!</h3>".
			   "Set the correct read/write permissions to the password file. <br />
			    Find the password file in the /files/ directory and CHMOD it to 640.<br /><br />
			    If this doesn't work, use <a href='http://krneky.com/forum'>this forum</a>.";
	   exit;
   }
   fwrite($file, $password);
   fclose($file);
   }

   function extraSettings()
   {
   global $description, $keywords, $title, $slogan, $copyright, $menu;
   echo "<div class='settings'>
<h3>Extra Settings</h3>
<a href='javascript:showhide()'>Cick here to open/close settings</a> <br /><br />
<div id=hide style='display: none;'>
<ul class='linkss'>
<b>Add Page: (in a new row) and <a href='javascript:location.reload(true);'>click here to refresh the page</a>.</b><br />
<div id='change'><span id='menu' class='editText'>$menu</span></div>
</ul>
<ul class='linkss'>
<b>Title:</b><br />
<div id='change'><span id='title' class='editText'>$title</span></div>
</ul>
<ul class='linkss'>
<b>Slogan:</b><br />
<div id='change'><span id='slogan' class='editText'>$slogan</span></div>
</ul>
<ul class='linkss'> 
<b>Meta Description:</b><br />
<div id='change'><span id='description' class='editText'>$description</span></div>
</ul>
<ul class='linkss'>
<b>Meta Keywords:</b><br />
<div id='change'><span id='keywords' class='editText'>$keywords</span></div>
</ul>
<ul class='linkss'>
<b>Copyright:</b><br />
<div id='change'><span id='copyright' class='editText'>$copyright</span></div>
</ul>
<br />
If you want to use a copyright sign: © - Simply copy this into your footer: &#38;&#99;&#111;&#112;&#121;&#59;
</div></div>";
   }
?>

This is the edittext.php

<?php

   session_start();

   function getSlug( $page ) {
   $page = strip_tags( $page );
   preg_match_all( "/([a-z0-9A-Z-_]+)/", $page, $matches );
   $matches = array_map( "ucfirst", $matches[0] );
   $slug = implode( "-", $matches );
   return $slug;
   }

   $fieldname = $_REQUEST['fieldname'];
   
   $encrypt_pass = @file_get_contents("files/password");
   if ($_COOKIE['wondercms']!=$encrypt_pass)
   {
   echo "You must login before using this function!";
   exit;
   }
   
   $content = rtrim(stripslashes($_REQUEST['content']));
   // if to only allow specified tags
   if($fieldname=="title")    $content = strip_tags($content);
   else	$content = strip_tags($content,"<audio><source><embed><p><h1><h2><h3><h4><h5><h6><a><img><u><i><em><strong><b><strike><center><pre>");

   $content = trim($content);
   $content = nl2br($content);

   if(!$content) $content = "Please be sure to enter some content before saving.";

   $content = preg_replace ("/%u(....)/e", "conv('\\1')", $content);

   if($fieldname>0 && $fieldname<4) $fname = "attachment$fieldname";
   else $fname = $fieldname;
   $file = @fopen("files/$fname.txt", "w");
   if(!$file)
   {
echo "<h2 class='wrong'>*Error* - unable to open $fieldname</h2><h3>But don't panic!</h3>".
"Just set the correct read/write permissions to the files folder.<br/>
Find the /files/ folder and CHMOD it to 751.<br /><br />
If this still gives you problems, open up the /files/ folder, select all files and CHMOD them to 640.<br /><br />
If this doesn't work, use <a href='http://krneky.com/forum'>this forum</a>.";
   exit;
   }
   fwrite($file, $content);
   fclose($file);

   echo $content;
   
   // convert udf-8 hexadecimal to decimal
   function conv($hex)
   {
   $dec = hexdec($hex);
   return "&#$dec;";
   }
?>

[attachment deleted by admin]

r0b · June 4, 2011

Bumping this.

WebStyles · June 4, 2011

sounds like an encoding issue. where's your <head> section. are you defining utf-8 on all pages?

r0b · June 5, 2011

Using

<meta http-equiv="content-type" content="text/html; charset=utf-8" />

on all pages, check the deafult.php to see the template if interested.

Another day, another bump.

r0b · June 6, 2011

1 day bump.

WebStyles · June 6, 2011

it's quite a lot of code, sod I can't really re-create your situation right now because I'm at work, but I can suggest a few more things:

1. check if txt files were also created with UTF-8

2. you can use file_put_contents() to save to the files instead of fopen/fwrite and fclose.

3. maybe track the exact situation where the error occurs (i.e. operating system + browser version) to see if there's a pattern and narrow down the problem.

r0b · June 6, 2011

it's quite a lot of code, sod I can't really re-create your situation right now because I'm at work, but I can suggest a few more things:

1. check if txt files were also created with UTF-8

2. you can use file_put_contents() to save to the files instead of fopen/fwrite and fclose.

3. maybe track the exact situation where the error occurs (i.e. operating system + browser version) to see if there's a pattern and narrow down the problem.

No problem thanks for having a go.

The files create a problem no matter what encoding type they're saved with. Using fopen or fwrite doesn't help either.

I don't think it's a matter or OS or browser, because special characters only work on some servers, so maybe getting the server information and comparing it with another one would be good.

For example the installation of this works on a localhost without a problem. (with same OS and browser).

Any other suggestions?

xyph · June 6, 2011

Are the PHP files encoded in UTF-8?

If that doesn't work, try forcing a BOM at the start of each text file like here

http://www.codingforums.com/showthread.php?t=129270

If THAT doesn't work, try utf8_encode()

http://php.net/manual/en/function.utf8-encode.php

IF THAAAAT doesn't work, use the iconv library

http://php.net/manual/en/book.iconv.php

If you're still hooped, you may want to start recoding with the problem characters in mind

r0b · June 7, 2011

Are the PHP files encoded in UTF-8?

If that doesn't work, try forcing a BOM at the start of each text file like here

http://www.codingforums.com/showthread.php?t=129270

If THAT doesn't work, try utf8_encode()

http://php.net/manual/en/function.utf8-encode.php

IF THAAAAT doesn't work, use the iconv library

http://php.net/manual/en/book.iconv.php

If you're still hooped, you may want to start recoding with the problem characters in mind

I tired your first option and saving all the files in UTF-8. This didn't solve the problem.

Then I tried the forcing BOM:

function writeUTF8File($fname,$content) { 
        $file=fopen("files/$fname.txt", "w"); 
	fwrite($file, pack("CCC",0xef,0xbb,0xbf)); 
        fwrite($file,$content); 
        fclose($file); 
} 
writeUTF8File($fname,$content);

Didn't solve the case either. Maybe I'm not using it right.

A weird case when using utf-8 and playing with

$content = mb_convert_encoding($content, 'UTF-8');

It works for German letters like üöë... etc, but not for chinese letters or Slovenian letters like š č ž.

This code also works for German: (it would work with chinese or š č ž letters).

$ent =array('À'=>'À', 'à'=>'à', 'Á'=>'Á', 'á'=>'á', 'Â'=>'Â', 'â'=>'â', 'Ã'=>'Ã', 'ã'=>'ã', 'Ä'=>'Ä', 'ä'=>'ä', 'Å'=>'Å', 'å'=>'å', 'Æ'=>'Æ', 'æ'=>'æ', 'Ç'=>'Ç', 'ç'=>'ç', 'Ð'=>'Ð', 'ð'=>'ð', 'È'=>'È', 'è'=>'è', 'É'=>'É', 'é'=>'é', 'Ê'=>'Ê', 'ê'=>'ê', 'Ë'=>'Ë', 'ë'=>'ë', 'Ì'=>'Ì', 'ì'=>'ì', 'Í'=>'Í', 'í'=>'í', 'Î'=>'Î', 'î'=>'î', 'Ï'=>'Ï', 'ï'=>'ï', 'Ñ'=>'Ñ', 'ñ'=>'ñ', 'Ò'=>'Ò', 'ò'=>'ò', 'Ó'=>'Ó', 'ó'=>'ó', 'Ô'=>'Ô', 'ô'=>'ô', 'Õ'=>'Õ', 'õ'=>'õ', 'Ö'=>'Ö', 'ö'=>'ö', 'Ø'=>'Ø', 'ø'=>'ø', 'Œ'=>'Œ', 'œ'=>'œ', 'ß'=>'ß', 'Þ'=>'Þ', 'þ'=>'þ', 'Ù'=>'Ù', 'ù'=>'ù', 'Ú'=>'Ú', 'ú'=>'ú', 'Û'=>'Û', 'û'=>'û', 'Ü'=>'Ü', 'ü'=>'ü', 'Ý'=>'Ý', 'ý'=>'ý', 'Ÿ'=>'Ÿ', 'ÿ'=>'ÿ','&' =>'&');
$content=strtr($content, $ent);

Is there any other universal code that will do the same as the code above, only for all languages?

xyph · June 8, 2011

http://webdesign.about.com/od/localization/l/blhtmlcodes-cz.htm

r0b · June 9, 2011

http://webdesign.about.com/od/localization/l/blhtmlcodes-cz.htm

This would solve the specific language problem, but let's say a person from China would want to use their characters.

I'm still not sure why

$content = mb_convert_encoding($content, 'UTF-8');

works for ü (umlauts and other special characters but not for lets say Chinese ones.

xyph · June 9, 2011

Here's this from the manual.

For those who, like me, lost a lot of minutes (hours) to understand why fwrite doesn't create a real utf-8 file, here's the explanation I've found :

I tried to do something like this :

<?php

$myString = utf8_encode("Test with accents éèàç");

$fh=fopen('test.xml',"w");

fwrite($fh,$myString);

fclose($fh);

?>

For a mysterious reason, the resulted file shows the accent without the utf-8 conversion.

I tried the binary, mode, etc. etc. And finally I've found it :

It seems that fwrite NEEDS to have the utf8_encode function INSIDE its parameters like this, to understand it must create a non-text only file :

<?php

$myString = "Test with accents éèàç";

$fh=fopen('test.xml',"w");

fwrite($fh,utf8_encode($myString));

fclose($fh);

?>

Hope this will help

If the PHP file is UTF-8, the HTML meta tag is declaring it's UTF-8, and the text file you create is UTF-8, you may need to start from scratch. Check this page out for issue with string functions and UTF-8

http://www.phpwact.org/php/i18n/utf-8

Sign In

Special characters work only on some servers only. (flat file cms)

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation

Important Information