htmlentities running when not wanted

[freelance84]

 

Creating a session in one script, only to find later that the content appears to have had htmlentities run on it...

 

Order of events:

1) Creating a $_SESSION['test'] with the following content: "£ £" in script one.

 

2) print_r($_SESSION['test']) at the end of script one and it displays "£ £"

 

3) next action - move onto another page created by php, line 2 of said script is print_r($_SESSION['test']), the result of which is "£ £"

 

There isn't anywhere that the session in question has had htmlentities run on it (not even on another script anywhere on the site), but it sill looks like "£ £".

 

Are there any php system settings which might be causing this?

[Buddski]

That looks more like a character encoding issue.

[freelance84]

That looks more like a character encoding issue.

 

True, however that doesnt seem to explain:

1) '£ £' into $_SESSION

2)print_r($_SESSION) to make sure it is in ok, pass.

3)access $_SESSION from another script and the content has been altered

 

 

I really have absolutely no idea how to correct this.

 

On the assumption that it may be a php config setting, i have tried to use htmlspecialchars_decode and html_entity_decode on the content of the $_SESSION but neither have any affect.

 

If anyone has had any issues with anything like this, I would be very appreciative to here about them and/or how you managed to resolve the issue.

 

Thanks,

 

John

[AbraCadaver]

That looks more like a character encoding issue.

 

True, however that doesnt seem to explain:

1) '£ £' into $_SESSION

2)print_r($_SESSION) to make sure it is in ok, pass.

3)access $_SESSION from another script and the content has been altered

 

 

I really have absolutely no idea how to correct this.

 

On the assumption that it may be a php config setting, i have tried to use htmlspecialchars_decode and html_entity_decode on the content of the $_SESSION but neither have any affect.

 

If anyone has had any issues with anything like this, I would be very appreciative to here about them and/or how you managed to resolve the issue.

 

Thanks,

 

John

 

Most likely the pages don't have the same character encoding.  Try making sure they are both UTF-8.

[Pikachu2000]

If it has anything to do with htmlentities, you'll see £ in the markup when you view the html source of the page.

[freelance84]

The text placed into the $_SESSION is later placed into a database.

 

After placing the text into the $_SESSION, it is never touched again (up until it is about to enter the db, at which point i run mysql_real_escape_string).

 

However before the $_SESSION is accessed for the db it is changing somehow.

 

 

Most likely the pages don't have the same character encoding.  Try making sure they are both UTF-8.

 

All the pages in question have the following encoding:

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />

 

Is this the character coding you meant? Or is there a character coding that can apply in the php config?

[freelance84]

The full process:

 

1) user enters text into a textarea

 

2) JS sends the text in the post to the server via AJAX

 

3) php stores the text into a session, and asks the user a question about their text content

 

4) Upon answering the question from step3 a normal html form is submitted to the server.

 

5) Depending on the answer of the question at step3, the text previously stored into $_SESSION may or my not get entered into the db.

 

It somewhere between step4 and step5 that the session changes somehow.

 

 

[cs.punk]

The full process:

 

1) user enters text into a textarea

 

2) JS sends the text in the post to the server via AJAX

 

3) php stores the text into a session, and asks the user a question about their text content

 

4) Upon answering the question from step3 a normal html form is submitted to the server.

 

5) Depending on the answer of the question at step3, the text previously stored into $_SESSION may or my not get entered into the db.

 

It somewhere between step4 and step5 that the session changes somehow.

 

Why don't you output the value of $_SESSION['test'] after each time you do something to it.

Eventually you will get to the bottom of it.

[freelance84]

Yea we did, quite a lot but nothing seem to be showing anything.

 

Well, after some testing we found that the reason it was messing up: somehow '%c2' was being adding to the front of the '%A3'

 

The following work around fixes things for the more common symbols used in english texts:

print_r($_SESSION['adNewCom'][0]);
//NB this is just test variable names
$aaa = urlencode($_SESSION['adNewCom'][0]);//NB this session just happened to be the one we were working on.
$aaa = str_replace("%C2", "", $aaa);//NB this of course means that %C2 cant be used, going to change the script to replace %c2% instead of just %C2
$bbb = urldecode($aaa);
print_r($bbb);

 

The above printed the following:

"Array ( [0] =>pound sign = £ ) pound sign = £"

 

There are other special characters which do not add %c2 at the beginning but add something to the end instead. However as they are very unique we have decided to ignore them for the time being, and return to the issue at a later date.

 

The seriously weird thing about all of this is that if we do not store the info into a session this doesn't appear to happen (ie if the data is stored on the html source or into a temp table). I'm assuming this may be some odd setting on the server?

 

 

 

If anyone else tries getting some text into a session via AJAX, then accessing that same text later from a different script and doesnt have the same issue i would love to know as this would indicate there is something odd going on somewhere that i have missed out.

 

Much thanks,

 

John

[freelance84]

 

Could somebody please tell me what they get when running htmlentities on '£'

 

I get '£'.

 

This can't be right can it, when printing, the source even shows '£'

[Buddski]

Code View

$str = '£';
echo $str.'<br/>';
echo htmlentities($str);

 

Browser

£
£

 

Source

£<br/>£

 

Hope this helps..

[voip03]

can  you use utf8_decode ?

[xyph]

This is an encoding issue for sure.

 

This works fine on my machine. File is encoded using UTF-8. Didn't test with 8859.

 

<?php 

session_start();

if( isset($_GET['destroy']) )
$_SESSION = array();

if( !empty($_POST['pound']) && $_POST['pound'] == '£' )
$_SESSION['pound'] = $_POST['pound'];

?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>The pound sign</title>
</head>
<body>
<div><form method="post" action="<?php echo $_SERVER['SCRIPT_NAME']; ?>">
	<ul style="list-style-type: none;">
		<li><label>Put pound sign here: <input type="text" name="pound"></label></li>
		<li><input type="submit"></li>
	</ul>
</form></div>
<div>
	<a href="javascript:location.reload(true)">refresh</a> or <a href="?destroy=1">restart</a>
</div>
<div>
	<pre><?php echo (!empty($_SESSION) ? print_r($_SESSION) : ''); ?></pre>
</div>
</body>
</html>

[freelance84]

Thanks xyph, i tried playing around with your encodings but nothing changed.

 

I think I have narrowed it down to a problem with AJAX.

 

'ä more testers ££ §'

 

The above I tested with a standard form. The string I entered into a textarea, submitted the form, ran htmlentities on it and entered it into the db.

When the info was pulled out all was fine and printed onto the page all was fine, proper characters used in the source and represented by the browser perfectly.

The following were used:

DOCTYPE:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

META:

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />

 

 

I then tested the AJAX: 'ä' to the server with a regular $_GET AJAX call:

function replaceCommentF1(subj){
   var cell = document.getElementById('newCommentText').value;
   xmlsetup() /*for ie*/
   /*Create the function to be executed when the server response is ready*/
   xmlhttp.onreadystatechange=function(){
      if (xmlhttp.readyState==4 && xmlhttp.status==200){
        alert(xmlhttp.responseText);
        document.getElementById('confirmAdd').innerHTML = xmlhttp.responseText;
      }
     }
     /*Send the request off to a file on the server*/
   xmlhttp.open("GET","http://www.mysite.net/commonAdding.php?repF="+cell,true);
   xmlhttp.send();
}

 

The processor waiting at the server:

if(isset($_POST['repF']))
{
  $comment = htmlentities($_POST['repF']);
  echo $_POST['repF'];
  echo $comment;
  exit();
}

 

The result:

The alert from the response text: 'ä ä'

Where is the 'Ã' coming from?

 

The content of the div from the response text: 'ä ä'

Again where is 'ä' coming from?

 

SUMMARY:

htmlentities is working fine.

When running htmlentities on the server and then sending the result back to the client through the response text, something goes wrong.

 

 

PS/

One issue I had not realised when bug finding, FireBug doesn't appear to show the actual source of an element. If you inspect element on a pound sign in a span for example, it will show you <span>£</span> where the source is actually the html special character. As a result it is very hard to bug find special chracter issues on an AJAX based problem. If anyone knows a way to make firebug view the actual html source, i would love to know

 

 

 

[xyph]

I STILL say it's an encoding issue.

 

This script functions as expected.

 

<?php

session_start();

if( isset($_GET['ajaxCall']) ) {

// AJAX RESPONSE
if( isset($_GET['pound']) ) {
	$_SESSION['pound'] = $_GET['pound'];
	echo 'true';
} else
	echo 'false';

} else {

if( isset($_GET['destroy']) )
	$_SESSION = array();

?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>AJAX Response</title>
<script type="text/javascript">
	function doAjax() {
		var input = document.getElementById('pound');
		var response = document.getElementById('response');
		var ajax = new XMLHttpRequest();
		ajax.onreadystatechange = function() {
			if( ajax.readyState == 4 && ajax.status == 200 ) {
				if( ajax.responseText == 'true' ) {
					response.innerHTML = 'AJAX Success. <a href="<?php echo $_SERVER['SCRIPT_NAME']; ?>">Refresh the page</a>';
				} else {
					response.innerHTML = 'AJAX Failed (' + ajax.responseText + ')';
				}
			}
		}
		ajax.open( 'GET', '<?php echo $_SERVER['SCRIPT_NAME']; ?>?ajaxCall=1&pound=' + input.value, true );
		ajax.send();
	}
</script>
</head>
<body>
<div><form method="post" action="<?php echo $_SERVER['SCRIPT_NAME']; ?>">
	<ul style="list-style-type: none;">
		<li><label>Put pound sign here: <input type="text" name="pound" id="pound"></label></li>
		<li><input type="button" value="Submit" onclick="doAjax()"></li>
	</ul>
</form></div>
<div>
	<a href="?destroy=1">restart</a><br>
	<span id="response"></span>
</div>
<div>
	<pre><?php echo (!empty($_SESSION) ? print_r($_SESSION) : ''); ?></pre>
</div>
</body>
</html>
<?php } ?>

[xyph]

Quick modification that echo's the input back on to the page. Same good result.

 

<?php

session_start();

if( isset($_GET['ajaxCall']) ) {

// AJAX RESPONSE
if( isset($_GET['pound']) ) {
	$_SESSION['pound'] = $_GET['pound'];
	echo $_GET['pound'];
}

} else {

if( isset($_GET['destroy']) )
	$_SESSION = array();

?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>AJAX Response</title>
<script type="text/javascript">
	function doAjax() {
		var input = document.getElementById('pound');
		var response = document.getElementById('response');
		var ajax = new XMLHttpRequest();
		ajax.onreadystatechange = function() {
			if( ajax.readyState == 4 && ajax.status == 200 ) {
					response.innerHTML = 'AJAX Success. <a href="<?php echo $_SERVER['SCRIPT_NAME']; ?>">Refresh the page</a> ('+ ajax.responseText + ')';
			}
		}
		ajax.open( 'GET', '<?php echo $_SERVER['SCRIPT_NAME']; ?>?ajaxCall=1&pound=' + input.value, true );
		ajax.send();
	}
</script>
</head>
<body>
<div><form method="post" action="<?php echo $_SERVER['SCRIPT_NAME']; ?>">
	<ul style="list-style-type: none;">
		<li><label>Put pound sign here: <input type="text" name="pound" id="pound"></label></li>
		<li><input type="button" value="Submit" onclick="doAjax()"></li>
	</ul>
</form></div>
<div>
	<a href="?destroy=1">restart</a><br>
	<span id="response"></span>
</div>
<div>
	<pre><?php echo (!empty($_SESSION) ? print_r($_SESSION) : ''); ?></pre>
</div>
</body>
</html>
<?php } ?>

[freelance84]

Hey, cool but that is not running htmlentities. When you try the following, it fails:

 

<?php

session_start();

if( isset($_GET['ajaxCall']) ) {

// AJAX RESPONSE
if( isset($_GET['pound']) ) {
	$_SESSION['pound'] = $_GET['pound'];
	echo htmlentities($_GET['pound']);
}

} else {

if( isset($_GET['destroy']) )
	$_SESSION = array();

?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>AJAX Response</title>
<script type="text/javascript">
	function doAjax() {
		var input = document.getElementById('pound');
		var response = document.getElementById('response');
		var ajax = new XMLHttpRequest();
		ajax.onreadystatechange = function() {
			if( ajax.readyState == 4 && ajax.status == 200 ) {
					response.innerHTML = 'AJAX Success. <a href="<?php echo $_SERVER['SCRIPT_NAME']; ?>">Refresh the page</a> ('+ ajax.responseText + ')';
			}
		}
		ajax.open( 'GET', '<?php echo $_SERVER['SCRIPT_NAME']; ?>?ajaxCall=1&pound=' + input.value, true );
		ajax.send();
	}
</script>
</head>
<body>
<div><form method="post" action="<?php echo $_SERVER['SCRIPT_NAME']; ?>">
	<ul style="list-style-type: none;">
		<li><label>Put pound sign here: <input type="text" name="pound" id="pound"></label></li>
		<li><input type="button" value="Submit" onclick="doAjax()"></li>
	</ul>
</form></div>
<div>
	<a href="?destroy=1">restart</a><br>
	<span id="response"></span>
</div>
<div>
	<pre><?php echo (!empty($_SESSION) ? print_r($_SESSION) : ''); ?></pre>
</div>
</body>
</html>
<?php } ?>

 

 

 

//

Have I just missed a monumental thing here, and have been doing for over a year? Are you not meant to run html entities in scenarios like this?

[xyph]

You need to define a character encoding when using htmlentities

 

Use htmlspecialchars anyways. htmlentities() is redundant, and does more than it needs to in order to prevent XSS attacks. htmlspecialchars() only changes what's needed, and is safe by default when using most of the popular character encodings.

 

<?php

session_start();

if( isset($_GET['ajaxCall']) ) {

// AJAX RESPONSE
if( isset($_GET['pound']) ) {
	$_SESSION['pound'] = $_GET['pound'];
	echo htmlentities( $_GET['pound'], ENT_COMPAT, 'UTF-8' );
	// or
	// echo htmlspecialchars( $_GET['pound'] );
}

} else {

if( isset($_GET['destroy']) )
	$_SESSION = array();

?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>AJAX Response</title>
<script type="text/javascript">
	function doAjax() {
		var input = document.getElementById('pound');
		var response = document.getElementById('response');
		var ajax = new XMLHttpRequest();
		ajax.onreadystatechange = function() {
			if( ajax.readyState == 4 && ajax.status == 200 ) {
					response.innerHTML = 'AJAX Success. <a href="<?php echo $_SERVER['SCRIPT_NAME']; ?>">Refresh the page</a> ('+ ajax.responseText + ')';
			}
		}
		ajax.open( 'GET', '<?php echo $_SERVER['SCRIPT_NAME']; ?>?ajaxCall=1&pound=' + input.value, true );
		ajax.send();
	}
</script>
</head>
<body>
<div><form method="post" action="<?php echo $_SERVER['SCRIPT_NAME']; ?>">
	<ul style="list-style-type: none;">
		<li><label>Put pound sign here: <input type="text" name="pound" id="pound"></label></li>
		<li><input type="button" value="Submit" onclick="doAjax()"></li>
	</ul>
</form></div>
<div>
	<a href="?destroy=1">restart</a><br>
	<span id="response"></span>
</div>
<div>
	<pre><?php echo (!empty($_SESSION) ? print_r($_SESSION) : ''); ?></pre>
</div>
</body>
</html>
<?php } ?>

[freelance84]

I can't believe that was it! And i can't believe I over looked that for an entire day!!

 

xyph, legendary!!

 

Thank you very much, (and everyone else of course)!!!