upvote
Any use of sum types would have prevented this immediately.

The whole thing exists because APIs that return a NULL value that "nobody checks", plus a string that you shouldn't use if you actually did check the null value.

In languages with sum types, this cannot happen.

Either you get your hash, or no result at all. You couldn't get eg NULL and *.

reply
C has unions. C has sum types. People have been doing tagged unions in C for longer than rust has been a language
reply
I don't think I've ever seen a C function that uses unions for this purpose though?

I can think of two common patterns that I've seen frequently:

* Return NULL for errors (sometimes with a global errorno() function).

* Return an integer error with any return values as out parameters.

So if it is possible to improve, why doesn't anyone do it?

reply
Yet the authors of crypt did not use them. The C language is not designed in a way to encourage people to write code that correctly uses tagged unions (or nullable pointers for that matter).
reply
Where are you going with those goal posts?
reply
Nowhere. It's the same argument as upthread. A language that forces downstream consumers of an API to correctly deal with all possible outcomes would have prevented this.
reply
Nope.

Because in fact all the changes were about changing the function signatures. The function signature is enforced by the linker. The semantic of strings that go through isn't.

Returning 2 values is possible in C (via pointers input parameters). The main issue here is that libc is a strong API contract based on C function signatures, that nobody can change, only augment with new symbols. Such strong API compatibility constraints happen in any programming language, and workaround to fit new behaviours in old APIs happens everywhere.

reply
I don't think nh2 is saying that this is how the "issue" should've been solved here, but that it's a fatal flaw of C itself (yes I know, unions, but no one uses them).

But also, its not like there were many alternative languages when the POSIX standards were designed so it's fair that it works like this.

reply
deleted
reply
C's lack of all of

- sum/union types

- multiple return values/tuple types

- exceptions

means that most ways of returning an error from a function are not available in C. Because of this, many C APIs, including `crypt` in this case, resort to in-band error signalling by using a sentinel value (like NULL, a negative integer, "*" etc.).

Most other languages lack one or two of the above, but very few lack all three, and so most have developed a different culture around error reporting (Result in Rust, multiple return values in Go, exceptions in Java, Python etc.).

reply
One of the small but important significances of Rust is that it very often gets to have the same machine code as a correct "in-band error signalling" strategy from C but with source code that the maintainers can actually understand.

And really often when your "in-band error signalling" in Rust is more expensive you realise (perhaps to your horror) that your C wouldn't be correct, so you just dodged a bullet. For example Option<OwnedFd> is, like OwnedFd, a 32-bit value, Unix has always reserved -1 to mean invalid, so Rust can treat that as a niche. But the Windows equivalent Option<OwnedHandle> can't do this because, well, cue Raymond Chen: https://devblogs.microsoft.com/oldnewthing/20040302-00/?p=40...

reply
The first 2 it has, and the 3rd, you really don't want.
reply
C has sum types/unions? What are you talking about? You don't need multiple return. You can use a struct and it's the same thing.
reply
not in a way that forces the user to actually handle them properly. C is a weakly typed language and as such is pretty badly suited to systems programming.
reply
For the other part of the issue (the truncation in SHA512), in Rust you could've had:

    struct Input<'a>(&'a [u8]);
    

    impl<'a> Input<'a> {
      fn new(input: &'a [u8]) -> Result<Self, Error> {
        if input.len() >= 256 {
           return Err(Errro...);
        }
        
        Ok(Self(input))
      }
    }
Now the truncation is a terrible idea in the first place, but at least with this it would have been entirely obvious to the caller that it was there.
reply
Wouldn't Rust's Result or Go's error checking prevent that?

While not mandatory, a person would probably use it instead of the * hack.

Simply because the calling side wouldn't have an option to forget the check like with the NULL case

reply
Rust and go still make it pretty easy to not check results if you don’t care about the success value (lots of code just does `let _ = foo()`), but if you do want the value (like with a hashing function), Rust makes it really hard to not check the error too. There’s no syntax to say “assume this worked and give me the success value, including whatever garbage is there if it’s not set”:

- result.unwrap() will panic if it failed

- result.ok() throws away the error but gives you a None value instead (still can’t use it as if it’s a success)

- matching on the Ok variant and doing nothing on the Err variant, you only get to use the success value inside the Ok arm

Etc etc.

The problem is when the compiler says “NULL is a valid instance of T”, which go lets you do, and rust doesn’t (without using the unsafe keyword and intentionally casting a null pointer, which you don’t do by accident.) Go lets you write code like `var hash string; hash, _ = hashfn()` and it compiles and now you have null instead of a valid hash.

reply
> NULL is a valid instance of T

This by the way is Tony Hoare's famous "Billion Dollar Mistake"

reply