Php convert unicode to html

I have the following string:

$string = "★ This is some text ★";

I want to convert it to html entities:

$string = "★ This is some text ★";

The solution everyone is writing about:

htmlentities("★ This is some text ★", "UTF-8");

But htmlentities can't convert all unicodes to html entities. So it just gives me the same output as the input:

★ This is some text ★

I've also tried to combine this solution with both:

header('Content-Type: text/plain; charset=utf-8');

and:

mb_convert_encoding();

But this either prints and empty result, doesn't convert at all or wrongly converts the stars to:

Â

How to I convert ★ and all other unicode characters to the correct html entity?

(PHP 4, PHP 5, PHP 7, PHP 8)

htmlentitiesConvert all applicable characters to HTML entities

Description

htmlentities(
    string $string,
    int $flags = ENT_QUOTES | ENT_SUBSTITUTE | ENT_HTML401,
    ?string $encoding = null,
    bool $double_encode = true
): string

If you want to decode instead (the reverse) you can use html_entity_decode().

Parameters

string

The input string.

flags

A bitmask of one or more of the following flags, which specify how to handle quotes, invalid code unit sequences and the used document type. The default is ENT_QUOTES | ENT_SUBSTITUTE | ENT_HTML401.

Available flags constants
Constant NameDescription
ENT_COMPAT Will convert double-quotes and leave single-quotes alone.
ENT_QUOTES Will convert both double and single quotes.
ENT_NOQUOTES Will leave both double and single quotes unconverted.
ENT_IGNORE Silently discard invalid code unit sequences instead of returning an empty string. Using this flag is discouraged as it » may have security implications.
ENT_SUBSTITUTE Replace invalid code unit sequences with a Unicode Replacement Character U+FFFD (UTF-8) or &#FFFD; (otherwise) instead of returning an empty string.
ENT_DISALLOWED Replace invalid code points for the given document type with a Unicode Replacement Character U+FFFD (UTF-8) or &#FFFD; (otherwise) instead of leaving them as is. This may be useful, for instance, to ensure the well-formedness of XML documents with embedded external content.
ENT_HTML401 Handle code as HTML 4.01.
ENT_XML1 Handle code as XML 1.
ENT_XHTML Handle code as XHTML.
ENT_HTML5 Handle code as HTML 5.
encoding

An optional argument defining the encoding used when converting characters.

If omitted, encoding defaults to the value of the default_charset configuration option.

Although this argument is technically optional, you are highly encouraged to specify the correct value for your code if the default_charset configuration option may be set incorrectly for the given input.

The following character sets are supported:

Supported charsets
CharsetAliasesDescription
ISO-8859-1 ISO8859-1 Western European, Latin-1.
ISO-8859-5 ISO8859-5 Little used cyrillic charset (Latin/Cyrillic).
ISO-8859-15 ISO8859-15 Western European, Latin-9. Adds the Euro sign, French and Finnish letters missing in Latin-1 (ISO-8859-1).
UTF-8   ASCII compatible multi-byte 8-bit Unicode.
cp866 ibm866, 866 DOS-specific Cyrillic charset.
cp1251 Windows-1251, win-1251, 1251 Windows-specific Cyrillic charset.
cp1252 Windows-1252, 1252 Windows specific charset for Western European.
KOI8-R koi8-ru, koi8r Russian.
BIG5 950 Traditional Chinese, mainly used in Taiwan.
GB2312 936 Simplified Chinese, national standard character set.
BIG5-HKSCS   Big5 with Hong Kong extensions, Traditional Chinese.
Shift_JIS SJIS, SJIS-win, cp932, 932 Japanese
EUC-JP EUCJP, eucJP-win Japanese
MacRoman   Charset that was used by Mac OS.
''   An empty string activates detection from script encoding (Zend multibyte), default_charset and current locale (see nl_langinfo() and setlocale()), in this order. Not recommended.

Note: Any other character sets are not recognized. The default encoding will be used instead and a warning will be emitted.

double_encode

When double_encode is turned off PHP will not encode existing html entities. The default is to convert everything.

Return Values

Returns the encoded string.

If the input string contains an invalid code unit sequence within the given encoding an empty string will be returned, unless either the ENT_IGNORE or ENT_SUBSTITUTE flags are set.

Changelog

VersionDescription
8.1.0 flags changed from ENT_COMPAT to ENT_QUOTES | ENT_SUBSTITUTE | ENT_HTML401.
8.0.0 encoding is nullable now.

Examples

Example #1 A htmlentities() example

$str "A 'quote' is bold";// Outputs: A 'quote' is <b>bold</b>
echo htmlentities($str);// Outputs: A 'quote' is <b>bold</b>
echo htmlentities($strENT_QUOTES);
?>

Example #2 Usage of ENT_IGNORE

$str "\x8F!!!";// Outputs an empty string
echo htmlentities($strENT_QUOTES"UTF-8");// Outputs "!!!"
echo htmlentities($strENT_QUOTES ENT_IGNORE"UTF-8");
?>

See Also

  • html_entity_decode() - Convert HTML entities to their corresponding characters
  • get_html_translation_table() - Returns the translation table used by htmlspecialchars and htmlentities
  • htmlspecialchars() - Convert special characters to HTML entities
  • nl2br() - Inserts HTML line breaks before all newlines in a string
  • urlencode() - URL-encodes string

Sijmen Ruwhof

12 years ago

An important note below about using this function to secure your application against Cross Site Scripting (XSS) vulnerabilities.

When printing user input in an attribute of an HTML tag, the default configuration of htmlEntities() doesn't protect you against XSS, when using single quotes to define the border of the tag's attribute-value. XSS is then possible by injecting a single quote:

$_GET['a'] = "#000' onload='alert(document.cookie)";
?>

XSS possible (insecure):

$href = htmlEntities($_GET['a']);
print
""; # results in:
?>

Use the 'ENT_QUOTES' quote style option, to ensure no XSS is possible and your application is secure:

$href = htmlEntities($_GET['a'], ENT_QUOTES);
print
""; # results in:
?>

The 'ENT_QUOTES' option doesn't protect you against javascript evaluation in certain tag's attributes, like the 'href' attribute of the 'a' tag. When clicked on the link below, the given JavaScript will get executed:

$_GET['a'] = 'javascript:alert(document.cookie)';
$href = htmlEntities($_GET['a'], ENT_QUOTES);
print
"link"; # results in: link
?>

q (dot) rendeiro (at) gmail (dot) com

15 years ago

I've seen lots of functions to convert all the entities, but I needed to do a fulltext search in a db field that had named entities instead of numeric entities (edited by tinymce), so I searched the tinymce source and found a string with the value->entity mapping. So, i wrote the following function to encode the user's query with named entities.

The string I used is different of the original, because i didn't want to convert ' or ". The string is too long, so I had to cut it. To get the original check TinyMCE source and search for nbsp or other entity ;)

$entities_unmatched

= explode(',', '160,nbsp,161,iexcl,162,cent, [...] ');
$even = 1;
foreach(
$entities_unmatched as $c) {
    if(
$even) {
       
$ord = $c;
    } else {
       
$entities_table[$ord] = $c;
    }
   
$even = 1 - $even;
}

function

encode_named_entities($str) {
    global
$entities_table;$encoded_str = '';
    for(
$i = 0; $i < strlen($str); $i++) {
       
$ent = @$entities_table[ord($str{$i})];
        if(
$ent) {
           
$encoded_str .= "&$ent;";
        } else {
           
$encoded_str .= $str{$i};
        }
    }
    return
$encoded_str;
}
?>

realcj at g mail dt com

15 years ago

If you are building a loadvars page for Flash and have problems with special chars such as " & ", " ' " etc, you should escape them for flash:

Try trace(escape("&")); in flash' actionscript to see the escape code for &;

% = %25
& = %26
' = %27

function flashentities($string){
return
str_replace(array("&","'"),array("%26","%27"),$string);
}
?>

Those are the two that concerned me. YMMV.

phil at lavin dot me dot uk

12 years ago

The following will make a string completely safe for XML:

function philsXMLClean($strin) {
       
$strout = null;

        for (

$i = 0; $i < strlen($strin); $i++) {
               
$ord = ord($strin[$i]);

                if ((

$ord > 0 && $ord < 32) || ($ord >= 127)) {
                       
$strout .= "&#{$ord};";
                }
                else {
                        switch (
$strin[$i]) {
                                case
'<':
                                       
$strout .= '<';
                                        break;
                                case
'>':
                                       
$strout .= '>';
                                        break;
                                case
'&':
                                       
$strout .= '&';
                                        break;
                                case
'"':
                                       
$strout .= '"';
                                        break;
                                default:
                                       
$strout .= $strin[$i];
                        }
                }
        }

        return

$strout;
}
?>

hajo-p

8 years ago

The flag ENT_HTML5 also strips newline chars like \n with htmlentities while htmlspecialchars is not affected by that.

If you want to use nl2br on that string afterwards you might end up searching the problem like i did. This does not apply to other flags like e.g. ENT_XHTML which confused me.

Tested this with PHP 5.4 / 5.5 / 5.6-dev with same results, so it seems that this is an intended "feature".

ustimenko dot alexander at gmail dot com

10 years ago

For those Spanish (and not only) folks, that want their national letters back after htmlentities :)

protected function _decodeAccented($encodedValue, $options = array()) {
   
$options += array(
       
'quote'     => ENT_NOQUOTES,
       
'encoding'  => 'UTF-8',
    );
    return
preg_replace_callback(
       
'/&\w(acute|uml|tilde);/',
       
create_function(
           
'$m',
           
'return html_entity_decode($m[0], ' . $options['quote'] . ', "' .
           
$options['encoding'] . '");'
       
),
       
$encodedValue
   
);
}
?>

n at erui dot eu

10 years ago

html entities does not encode all unicode characters. It encodes what it can [all of latin1], and the others slip through. Љ is the nasty I use. I have searched for a function which encodes everything, but in the end I wrote this. This is as simple as I can get it. Consult an ansii table to custom include/omit chars you want/don't. I'm sure it's not that fast.

// Unicode-proof htmlentities.
// Returns 'normal' chars as chars and weirdos as numeric html entites.
function superentities( $str ){
    // get rid of existing entities else double-escape
    $str = html_entity_decode(stripslashes($str),ENT_QUOTES,'UTF-8');
    $ar = preg_split('/(?    foreach ($ar as $c){
        $o = ord($c);
        if ( (strlen($c) > 1) || /* multi-byte [unicode] */
            ($o <32 || $o > 126) || /* <- control / latin weirdos -> */
            ($o >33 && $o < 40) ||/* quotes + ambersand */
            ($o >59 && $o < 63) /* html */
        ) {
            // convert to numeric entity
            $c = mb_encode_numericentity($c,array (0x0, 0xffff, 0, 0xffff), 'UTF-8');
        }
        $str2 .= $c;
    }
    return $str2;
}

wd at NOSPAMwd dot it

10 years ago

Hi there,

after several and several tests, I figured out that dot:

- htmlentities() function remove characters like "à","è",etc when you specify a flag and a charset

- htmlentities() function DOES NOT remove characters like those above when you DO NOT specify anything

So, let's assume that..

$str

= "Hèèèllooo";$res_1 = htmlentities($str, ENT_QUOTES, "UTF-8");
$res_2 = htmlentities($str);

echo

var_dump($res_1); // Result: string '' (length=0)
echo var_dump($res_2); // string 'Hèèèllooo' (length=30)?>

I used this for a textarea content for comments. Anyway, note that using the "$res_2" form the function will leave unconverted single/double quotes. At this point you should use str_replace() function to perform the characters but be careful because..

$str

= "'Hèèèllooo'";$res_2 = str_replace("'","'",$str);
$res_2 = htmlentities($str);
echo
var_dump($res_2); // string '&#039;Hèèèllooo&#039;'$res_3 = htmlentities($str);
$res_3 = str_replace("'","'",$res_3);
echo
var_dump($res_3); // string ''Hèèèllooo'' --> Nice
?>

Hope it will helps you.

Regards,
W.D.

robin at robinwinslow dot co dot uk

11 years ago

htmlentities seems to have changed at some point between version 5.1.6 and 5.3.3, such that it now returns an empty string for anything containing a pound sign:

$ php -v
PHP 5.1.6 (cli) (built: May 22 2008 09:08:44)
$ php -r "echo htmlentities('£hello', null, 'utf-8');"
£hello
$

$ php -v
PHP 5.3.3 (cli) (built: Aug 19 2010 12:07:49)
$ php -r "echo htmlentities('£hello', null, 'utf-8');"
$

(Returns an empty string the second time)

Just a heads up.

Waygood

11 years ago

When putting values inside comment tags you should replace -- with -- too, as this would end your tag and show the rest of the comment.

2962051004 at qq dot com

3 years ago

/**
* 将中文转为Html实体
* Convert Chinese in HTML to entity
* Author QiangGe
* Mail
*
*/
$str = <<你好 world
EOT;

function

ChineseToEntity($str) {
return
preg_replace_callback(
       
'/[\x{4e00}-\x{9fa5}]/u', // utf-8
        // '/[\x7f-\xff]+/', // if gb2312
       
function ($matches) {
           
$json = json_encode(array($matches[0]));
           
preg_match('/\[\"(.*)\"\]/', $json, $arr);
           
/*
             * 通过json_encode函数将中文转为unicode
             * 然后用正则取出unicode
             * Turn the Chinese into Unicode through the json_encode function, then extract Unicode from regular.
             * I think this idea is seamless.
            */
           
return '&#x'. str_replace('\\u', '', $arr[1]). ';';
        },
$str
  
);
}

echo

ChineseToEntity($str);
// 你好 world

admin at wapforum dot rs

11 years ago

A useful little function to convert the symbols in the different inputs.
function ConvertSimbols($var, $ConvertQuotes = 0) {
if (
$ConvertQuotes > 0) {
$var = htmlentities($var, ENT_NOQUOTES, 'UTF-8');
$var = str_replace('\"', '', $var);
$var = str_replace("\'", '', $var);
} else {
$var = htmlentities($var, ENT_QUOTES, 'UTF-8');
}
return
$var;
}
?>

Usage with quotes for example message:

$message = ConvertSimbols($message);

Usage without quotes for example link:

$link = ConvertSimbols($link, 1);

Bassie (:

19 years ago

Note that you'll have use htmlentities() before any other function who'll edit text like nl2br().

If you use nl2br() first, the htmlentities() function will change < br > to <br>.

jake_mcmahon at hotmail dot com

18 years ago

This fuction is particularly useful against XSS (cross-site-scripting-). XSS makes use of holes in code, whether it be in Javascript or PHP. XSS often, if not always, uses HTML entities to do its evil deeds, so this function in co-operation with your scripts (particularly search or submitting scripts) is a very useful tool in combatting "H4X0rz".

Tom Walter

13 years ago

Note that as of 5.2.5 it appears that if the input string contains a character that is not valid for the output encoding you've specified, then this function returns null.

You might expect it to just strip the invalid char, but it doesn't.

You can strip the chars yourself like so:

iconv('utf-8','utf-8',$str);

You can combine that with htmlentities also:

$str = htmlentities(iconv('UTF-8', 'UTF-8//IGNORE', $str, ENT_QUOTES, 'UTF-8');

Should give you a string with htmlentities encoded to utf-8, and any unsupported chars stripped.

j2teamnnl at gmail dot com

4 months ago

The answer above is not correct for multiple languages like France
I had correct it
function xml_entities($strIn)
    {
        if (is_numeric($strIn)) {
            return $strIn;
        }
        $strOut = null;

        $arrStr = mb_str_split($strIn);
        foreach ($arrStr as $char) {
            $ord = mb_ord($char);

            if (($ord > 0 && $ord < 32) || ($ord >= 127)) {
                $strOut .= "&#{$ord};";
            }
            else {
                switch ($char) {
                    case '<':
                        $strOut .= '<';
                        break;
                    case '>':
                        $strOut .= '>';
                        break;
                    case '&':
                        $strOut .= '&';
                        break;
                    case '"':
                        $strOut .= '"';
                        break;
                    default:
                        $strOut .= $char;
                }
            }
        }

        return $strOut;
    }

Jeff

4 years ago

There is a feature when writing to XML using an AJAX call to PHP that rarely is mentioned. I struggled for many hours using htmlentities() because what was getting written to my XML document was not as expected. I naturally assumed that I should be converting my strings before writing them to XML to adhere to XML rules on illegal characters. To my surprise, when converting with htmlentities() or htmlspecialchars() and then writing to an XML file, the resulting ampersands get converted afterwards! Consider the following example:

$str = "I am cool" ;
$str = htmlentities($str) ;
?>

When you append $str to an XML element and save() the document, you would expect the XML document's source code to look something like this:

<b>I am cool</b>

But that is not what happens. The resulting ampersands get converted by PHP automatically to & and your source code ends up looking like this:

&lt;b&gt;I am cool&lt;/b&gt;

As you can see, this creates problems when trying to output the XML data back to HTML. It is important to remember that when writing to XML this way, special characters like ">" and "<"; PHP converts them automatically and there becomes no need to use htmlentities() in certain cases. I assume this feature is in place to aid with passing data through header queries, to avoid reserved characters conflicting with others in a header query (e.g. & or =). Now I understand this may not be the case with older versions of PHP and that this might be a feature of my version (PHP version 5.6.32). With older versions, I assume using htmlentities() or htmlspecialchars() is a must, as stated with previous notes here. Also I use the charset UTF-8 in my HTML and XML and am not sure if this also effects the results I get.

Anyway, I struggled for many hours with using htmlentities() to convert strings for XML writing and saving, when all I had to do was simply not use the function and let PHP convert my strings for me. I hope this helps because I would think I am not the only one who has struggled with this situation.

chris at ocproducts dot com

5 years ago

This function throws a warning on bad input even if ENT_SUBSTITUTE is set, so be prepared for this.

steve at mcdragonsoftware dot com

10 years ago

I'm glad 5.4 has xml support, but many of us are working with older installations, some of us still have to use PHP4. If you're like me you've been frustrated with trying to use htmlentites/htmlspecial chars with xml output. I was hoping to find an option to force numeric encoding, lacking that, I have written my own xmlencode function, which I now offer:

usage:

$string xmlencode( $string )

it will use htmlspecialchars for the valid xml entities amp, quote, lt, gt, (apos) and return the numeric entity for all other non alpha-numeric characters.

-------------------------------------------

if( !function_exists( 'xmlentities' ) ) {
    function
xmlentities( $string ) {
       
$not_in_list = "A-Z0-9a-z\s_-";
        return
preg_replace_callback( "/[^{$not_in_list}]/" , 'get_xml_entity_at_index_0' , $string );
    }
    function
get_xml_entity_at_index_0( $CHAR ) {
        if( !
is_string( $CHAR[0] ) || ( strlen( $CHAR[0] ) > 1 ) ) {
            die(
"function: 'get_xml_entity_at_index_0' requires data type: 'char' (single character). '{$CHAR[0]}' does not match this type." );
        }
        switch(
$CHAR[0] ) {
            case
"'":    case '"':    case '&':    case '<':    case '>':
                return
htmlspecialchars( $CHAR[0], ENT_QUOTES );    break;
            default:
                return
numeric_entity_4_char($CHAR[0]);                break;
        }       
    }
    function
numeric_entity_4_char( $char ) {
        return
"&#".str_pad(ord($char), 3, '0', STR_PAD_LEFT).";";
    }   
}
?>

h_guillaume at hotmail dot com

12 years ago

I use this function to encode all the xml entities and also all the &something; that are not defined in xml like ™
You can also decode what you encode with my decode function.
My function works a little like the htmlentities.
You can also add other string to the array if you want to exclude them from the encoding.

function xml_entity_decode($text, $charset = 'Windows-1252'){
   
// Double decode, so if the value was &trade; it will become Trademark
   
$text = html_entity_decode($text, ENT_COMPAT, $charset);
   
$text = html_entity_decode($text, ENT_COMPAT, $charset);
    return
$text;
}

function

xml_entities($text, $charset = 'Windows-1252'){
    
// Debug and Test
    // $text = "test & ™ &trade; abc ® &reg; -";

        // First we encode html characters that are also invalid in xml

$text = htmlentities($text, ENT_COMPAT, $charset, false);// XML character entity array from Wiki
    // Note: ' is useless in UTF-8 or in UTF-16
   
$arr_xml_special_char = array(""","&","'","<",">");// Building the regex string to exclude all strings with xml special char
   
$arr_xml_special_char_regex = "(?";
    foreach(
$arr_xml_special_char as $key => $value){
       
$arr_xml_special_char_regex .= "(?!$value)";
    }
   
$arr_xml_special_char_regex .= ")";// Scan the array for &something_not_xml; syntax
   
$pattern = "/$arr_xml_special_char_regex&([a-zA-Z0-9]+;)/";// Replace the &something_not_xml; with &something_not_xml;
   
$replacement = '&${1}';
    return
preg_replace($pattern, $replacement, $text);
}
?>

za at byza dot it

14 years ago

Trouble when using files with different charset?

htmlentities and html_entity_decode can be used to translate between charset!

Sample function:

function utf2latin($text) {
  
$text=htmlentities($text,ENT_COMPAT,'UTF-8');
   return
html_entity_decode($text,ENT_COMPAT,'ISO-8859-1');
}
?>

What is the use of HTML entities () function in PHP?

htmlentities() Function: The htmlentities() function is an inbuilt function in PHP that is used to transform all characters which are applicable to HTML entities. This function converts all characters that are applicable to HTML entities.

What is HTML entities ()?

An HTML entity is a piece of text ("string") that begins with an ampersand ( & ) and ends with a semicolon ( ; ). Entities are frequently used to display reserved characters (which would otherwise be interpreted as HTML code), and invisible characters (like non-breaking spaces).

How can I encode HTML code in PHP?

Encodes only double quotes. ENT_QUOTES - Encodes double and single quotes..
ENT_HTML401 - Default. Handle code as HTML 4.01..
ENT_HTML5 - Handle code as HTML 5..
ENT_XML1 - Handle code as XML 1..
ENT_XHTML - Handle code as XHTML..

How do I decode HTML entities?

The html_entity_decode() function converts HTML entities to characters. The html_entity_decode() function is the opposite of htmlentities().