undefined

points

[-]

Only know the discussion superficially, but from what I understood, the problem was as follows: Chinese Hanzi, Japanese Kanji and Korean Hanja scripts have the same historical roots (Han) and still look very similar when you write them — however, they are fully separate scripts with different use of letters and optical differences that are important.

However, when encoding them into Unicode, the consortium suddenly discovered the importance of being frugal with character IDs and only defined a single "Han" sequence of characters for all three scripts. You need to select a matching font to actually display the text in the script that's intended.

This makes using those characters an unnecessary hassle and also came over as a cultural faux-pas to some (essentially like encoding Greek Eta, Latin H and Cyrillic En as the same character, because they look similar and the scripts have a common history)

<s>It also looks a bit hypocritical when Unicode otherwise goes the opposite way and has the space to encode dozens of variants of the Latin alphabet or — as we see here — several categories of dashes.</s> (Edit: Han unification was done when Unicode was still 16 bit, so there were real space constraints — so no hypocrisy actually)

See here: https://en.wikipedia.org/wiki/Han_unification

by zahlman2 hours ago|

parent|

[-]

I honestly agree with the arguments they made to defend it, but then they tied themselves in loops trying to defend things like separate Greek and Cyrillic scripts and making superfine distinctions about how that isn't the same phenomenon. But they couldn't merge those scripts together now even if they wanted to, because of the stability policy. And the existence of the blackletter, small-caps, superscript, etc. characters seems entirely indefensible to me. All font rendering issues; it's clearly the same symbol with the same fundamental meaning.

Really, they needed to come up with the idea of variation selectors way earlier, and use them aggressively. And design something more consistent for character composition, and skip the precomposed characters (those really are glyphs representing a cluster; being able to treat the cluster as an atomic entity versus its components doesn't add actual semantic information to the text).