Htmlspecialchars
discussion before the release of 1.1.6.0
htmlspecialchars_unicode():
looking at the code I'm sure this will not work correctly. It wil "accept" numerical entity references but not named entity references - so those still won't work. (And actually, its operation doesn't have anything to do with
Unicode, only with entities - which
may encode Unicode characters but entities are not themselves Unicode.)
See my item 'Non-breaking space in forced links' in the SuggestionBox: this is one case that actually shows the function htmlspecialchars_unicode() does not work correctly!
Also, there is no option for the quote_style and charset parameters as in the PHP original, so we lose functionality here, too. It's probably better to have a "wrapper" function around the PHP one, which (after applying the PHP function, passing on the extra parameters) merely reverts all ampersands that are for any entity references (numerical or named); thus the wrapper function should accept all parameters the PHP function does. And since we are supposed to produce XHTML, ENT_QUOTES should probably be the default for quote_style; maybe also UTF-8 should be the default for charset?
Note: the INI code formatter used ENT_QUOTES and this has now disappeared (it was there for a reason!). But an entity used in
code should be visible as an entity, not as the character it encodes. Conclusion: a code formatter
should use the PHP function htmlspecialchars() so that any entities are "escaped".
Here's the solution: a new function htmlspecialchars_ent() to replace the proposed (beta) htmlspecialchars_unicode():
/**
* Wrapper around PHP's htmlspecialchars() which preserves (repairs) entity references.
*
* The function accepts the same parameters as htmlspecialchars() in PHP and passes them on
* to that function.
*
* One defaults here is different here from that in htmlspecialchars() in PHP:
* charset is set to UTF-8 so we're ready for UTF-8 support (and as long as we don't support
* that there should be no difference with Latin-1); on systems where the charset parameter
* is not available or UTF-8 is not supported this will revert to Latin-1 (ISO-8859-1).
*
* The function first applies htmlspecialchars() to the input string and then "unescapes"
* character entity references and numeric character references (both decimal and hexadecimal).
* Entities are recognized also if the ending semicolon is omitted at the end or before a
* newline or tag but for consistency the semicolon is always added in the output where it was
* omitted.
*
* NOTE:
* Where code should be rendered _as_code_ the original PHP function should be used so that
* entity references are also rendered as such instead of as their corresponding characters.
*
* @access public
* @since wikka 1.1.6.0
* @version 1.0
* @todo (later) support full range of situations where (in SGML) a terminating ; may legally
* be omitted (end, newline and tag are merely the most common ones).
*
* @param string $text required: text to be converted
* @param integer $quote_style optional: quoting style - can be ENT_COMPAT (default, escape
* only double quotes), ENT_QUOTES (escape both double and single quotes) or
* ENT_NOQUOTES (don't escape any quotes)
* @param string $charset optional: charset to use while converting; default UTF-8
* (overriding PHP's default ISO-8859-1)
* @return string converted string with escaped special characted but entity references intact
*/
function htmlspecialchars_ent
($text,
$quote_style=ENT_COMPAT,
$charset=
'UTF-8')
{
// define patterns
$alpha =
'[a-z]+';
# character entity reference
$numdec =
'#[0-9]+';
# numeric character reference (decimal)
$numhex =
'#x[0-9a-f]+';
# numeric character reference (hexadecimal)
$terminator =
';|(?=($|[\n<]|<))';
# semicolon; or end-of-string, newline or tag
$entitystring =
$alpha.
'|'.
$numdec.
'|'.
$numhex;
$escaped_entity =
'&('.
$entitystring.
')('.
$terminator.
')';
// execute PHP built-in function, passing on optional parameters
$output =
htmlspecialchars($text,
$quote_style,
$charset);
// "repair" escaped entities
// modifiers: s = across lines, i = case-insensitive
$output =
preg_replace('/'.
$escaped_entity.
'/si',
"&$1;",
$output);
// return output
return $output;
}
I created a test harness for it (I tested with
htmlspecialchars() in the Link() function in
wikka.php replaced by
htmlspecialchars_ent():
- [[HomePage word & notherword]] (to be escaped)
=> word & notherword
- [[JavaWoman Java Woman]] (alpha: no-breaking space)
=> Java Woman
- [[JavaWoman Ähnlich]] (alpha with uppercase)
=> Ähnlich
- no terminating ; before tag "<span style="color:blue;">blue</span>" (alpha: text, not link)
=> no terminating ; before tag "blue"
- [[CategoryDevelopmentTest test no terminating ; before end "]] (alpha)
=> test no terminating ; before end "
- [[Docs:FormattingRules <b>no ; before tag '</b>]] (numeric decimal)
=> <b>no ; before tag '</b>
- [[SandBox missing ; before ϵ
There are problems (now) with test cases 2, 3, 5, 6 and 7; with my proposed solution all testcases are converted correctly.
Note: The formatter does not recognize forced links with newline in description text (test case 7)! There is nothing wrong with having a newline inside a link description. => This requires small fix in wakka formatter (./formatters/wakka.php), as follows:
replace:
// forced links
// \S : any character that is not a whitespace character
// \s : any whitespace character
else if (preg_match("/^\[\[(\S*)(\s+(.+))?\]\]$/",
$thing,
$matches))
by:
// forced links
// \S : any character that is not a whitespace character
// \s : any whitespace character
else if (preg_match("/^\[\[(\S*)(\s+(.+))?\]\]$/s",
$thing,
$matches)) # s modifier: recognize forced links across lines
Finally: the call in ./formatters/ini.php should be restored to:
and in
./formatters/code.php we should also use
(i.e, add the ENT_QUOTES parameter).
All other calls to htmlspecialchars_unicode() (as well as the definition) should be replaced by htmlspecialchars_ent() above.
CategoryDevelopmentCore CategoryDevelopmentFormatters CategoryDevelopmentTest