Guys that grew up with the 8-bit micros did write the spec—the first design drafts of the standard started circulating in 1988, version 1.0 came out in 1991. Given IBM’s primary role in the whole thing, even parents of such guys could’ve been involved.

The problem was, they had to sell their invention to other guys that had grown up with 8-bit micros and in the meantime had gotten very attached to their specific 8-bit encodings, so they had to make a lot of compromises: first 128 characters match ASCII; first 256 characters match Latin-1; alphabets encoded in order; codepoint-to-codepoint transcoding from any (reasonable) legacy encoding; throw out enough stuff to not need more than 2^16 characters. They were also racing against time, because in the meanwhile ISO was cooking up 10646, the Universal Character Set, built around ISO 2022’s teletype-inspired architecture of charset switching, and that only got killed because software vendors essentially declared a boycott and pointed to Unicode instead.

The result came out rather awkward as well as fairly expensive, and a lot of the promises needed to be broken afterwards anyway. But they’d have no users had they had done things differently. (Meanwhile, the UCS as initially conceived lacked such advanced operations as string equality.)

I don't follow? If bit 5 == zero meant "this is whitespace", then white-space characters would take up fully half of the unicode space. There would be more than 2 billion of them, not 32. Seems a bit overkill...

That wouldn't work. As the "Nothing is set in stone" section points out, in Unicode 6.2 the MONGOLIAN VOWEL SEPARATOR went from space separator to a format character.