upvote
There was never any guarantee that case conversion is byte length equivalent. Except for pure ASCII, that's just been an incorrect assumption from the start.
reply
This only affects ISO-8859-1~16 encodings, Ruby only supports non-ASCII case folding for those and UTF. It might actually be the only character from that list of ISO encodings for which the length changes.
reply
It looks like the original author did actually try to account for this by allocating 20 more bytes than the input length.

From the blog post:

    The output buffer is allocated as the input length plus 20 bytes. Each ß adds one byte to the output, so an input with 21 of them overflows the buffer by one byte.
reply
Ah, so that's the real reason "ß" is now allowed in uppercase too (https://en.wikipedia.org/wiki/%C3%9F)? Reducing the risk of security issues by removing an edge case?
reply
The Unicode case mappings are covered by backwards-compatibility guarantees. "ß" is going to be mapped to "SS" forever.
reply
deleted
reply
It even seems the "more correct" uppercase form of ß is now ẞ, not SS.

> Traditionally, ⟨ß⟩ did not have a capital form, and was capitalized as ⟨SS⟩. Some type designers introduced capitalized variants. In 2017, the Council for German Orthography officially adopted a capital form ⟨ẞ⟩ as an acceptable variant, ending a long debate.[4] Since 2024 the capital has been preferred over ⟨SS⟩.[5]

reply
>ending a long debate

They did not. Participants from Liechtenstein and Switzerland don't even care about that letter (Swiss orthography does not make use of it), so they are not even a good place to begin such debate.

reply
The fact that they weren't participating in the debate doesn't mean the debate didn't end, let alone that it didn't begin.
reply
> It even seems the "more correct" uppercase form of ß is now ẞ, not SS.

But that is wrong. In german I only know of ß aka downcased version.

ẞ seems another character. I have no idea what it is, other than looking close to ẞ, but it most assuredly is not existing in the german language. There is simply no uppercased variant. 'ss' is "Doppel-s' - that is not ß. In german we have "das" (which is the), but it can be "das" or "dass" depending on the sentence, in particular after a ',' character. I often misspell it, since I prefer "das", when in reality sometimes one has to use "dass". But 'ß' is very different to that - to claim that 'ß' is equivalent to 'ss' is simply incorrect. I have no idea who that "Council for German Orthography" is, but clearly they don't know anything about German. Who put that Council in charge of anything? Are these drunks? The claim that the capital has been preferred, is also factually wrong. Who is writing all that?

Edit: Oh I see, you have that from Wikipedia.

The .pdf can be found here:

https://www.rechtschreibrat.com/DOX/RfdR_Amtliches-Regelwerk...

So from what I can see, this is only the:

"Leibniz-Institut für Deutsche Sprache"

That is not a unified authority for anything. It is totally unclear to me why the wikipedia article claims that this is legally binding for anyone. That's just factually incorrect. Is Wikipedia no longer proofreading?

reply
Who are you to decide what does and doesn't exist in the German language? You are no more authoritative than the Leibniz-Institut either.

Edit: The Leibniz-Institut is just the publisher. The author, the Rat für deutsche Rechtschreibung, is legitimized by the German (and six other countries') government.

The Duden is also seen by many as authoritative and also knows and allows the capital ß: https://www.duden.de/sprachwissen/rechtschreibregeln/doppel-...

Edit 2: It's also not illegal to use whatever made up rules you want. Nothing about this is legally binding anywhere.

reply
Wikipedia says ẞ is "preferred", which is plausible when an organization with an impressive name declares it preferable. Nobody says it's legally binding.

> I often misspell it, since I prefer "das", when in reality sometimes one has to use "dass".

It's confusing because both can mean "that".

"That is true." = "Das ist wahr."

"He said that it is true." = "Er sagte, dass es wahr ist."

reply