PHP is adding ? to unicode values

[Mr.n]

Dear all,

 

We are having a problem with php application. When we send unicode values the ? is automatically added to the text.

We tried to modify the php.ini directive: mbstring.substitute_character = "none" but nothing happened. Anyone can help please?

 

Thank you,

[effigy]

You've bad Unicode. See "Replacement character" here.

[Mr.n]

The strange thing is that everything was working normal !! and suddenly it started adding the ? character. How can i use character replacement.

[effigy]

"Suddenly"? Perhaps your input and/or code changed? Have you examined the data fully? A hex dump, perhaps?

[Mr.n]

Actually i have an sms application that creates automatically a url, here is the text making the problem: text=r%C3%A9r%C3%A9r%C3%A9

 

i did a debugging and now i am sure that php is adding something because the text does not have any added code!!

 

here is the code of the application:

 

    $URL .= "&text=".urlencode($sms_msg);
    $URL .= "&dlr-mask=31&dlr-url=".urlencode($kannel_param['playsms_web']."/plugin/gateway/kannel/dlr.php?type=%d&slid=$smslog_id&uid=$uid");
    $URL .= "&mclass=".$msg_type;
        }
        else
        {
    $URL = "/cgi-bin/sendsms?username=".urlencode($kannel_param['username'])."&password=".urlencode($kannel_param['password']);
    $URL .= "&from=".urlencode($sms_from_sender)."&to=".urlencode($sms_to)."&smsc=routeleb";
    if ($unicode == 1) {
      $URL .= "&charset=utf-8&coding=2";
    //  if (strlen($sms_msg) >= 70) $sms_msg = substr($sms_msg,0,69);
    } else if (strlen($sms_msg) >= 160) {
    // $sms_msg = substr($msg_msg,0,159);
    }
   $URL .= "&text=".urlencode($sms_msg);
   $URL .= "&dlr-mask=31&dlr-url=".urlencode($kannel_param['playsms_web']."/plugin/gateway/kannel/dlr.php?type=%d&slid=$smslog_id&uid=$uid");
    $URL .= "&mclass=".$msg_type;
        }
    $connection = fsockopen($kannel_param['bearerbox_host'],$kannel_param['sendsms_port'],&$error_number,&$error_description,60);
    if($connection)
    {
        socket_set_blocking($connection, false);
        fputs($connection, "GET $URL HTTP/1.0\r\n\r\n");

[effigy]

What's the value of $unicode?

 

This gives me "réréré":

 

<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
<pre>
<?php
$text = 'r%C3%A9r%C3%A9r%C3%A9';
echo urldecode($text);
?>
</pre>

[Mr.n]

i tried to encode a value as url and to send it. it arrived wrong with added ? . Do you have any idea if it can be caused by apache server

[Mr.n]

OKay i tried some arabic characters and discovered a new thing!! arabic chars appears as *? and latin chars appears like r? . here is an example !!!

thats what i write: réunionréunion تست

and it appears in unicode format as:  ?r?é?u?n?i?o?n?r?é?u?n?i?o?n ?*?3?*?

[Mr.n]

hey !! i still have one problem! i used the function utf8_decode and i succeeded in removing some extra character !! but the unicode values still appears as ? how to let php understand these unicode values ???

 

Thank you