Here they are in Futhark:
reply val concat [n] [m] 't : (xs: [n]t) -> (ys: [m]t) -> *[n + m]t
val matmul [n] [m] [l] 't : (xs: [n][m]t) -> (ys: [m][l]t) -> *[n][l]t
val head [n] 't : (x: [n]t) -> t
val filter [n] 'a : (p: a -> bool) -> (as: [n]a) -> *[]a
The Pyrefly type checker is starting to work on this kind of shape hinting - so far it only works on Torch but I believe the plan is for it to work with other array packages (eg. JAX, NumPy)
replySee also jaxtyping which, contrary to what its name might imply, covers JAX/PyTorch/NumPy/MLX/TensorFlow arrays and tensors.
replyShape functions and shape analysis are basically mundane infra in almost every ML compiler/language/DSL.
replyI didn't know that, thanks for sharing. It makes sense, but then it also makes me wonder why none of the deep learning libraries (Torch, Jax/NNX, Eigen etc...) make this information available. Instead, ML people all have their own schemes for tracking shape information, like commenting '# (b, n, t)' on every line, or suffixing shapes to variable names - and in my experience it's a common source of bugs.
reply> Torch, Jax
replyBoth of these "make it available". Just because people don't know how to use/find them doesn't mean they're not "available".
> Eigen
This is not an ML anything, it's a linear algebra library.
> like commenting '# (b, n, t)' on every line, or suffixing shapes to variable names
There's a difference between tracking shapes in the compiler and specifying shapes in the model.
You can do this with templates in C++ and generics in Rust I'm pretty sure. I think the Eigen C++ library supports this. (I have yet to do a linear algebra heavy Rust project, so I can't speak to the options that exist there.)
replyI'm talking about cases where the array size is not known at compile time. For example, say the user passes in a list of numbers as command line arguments. Then we have
reply argv: Vec<String, argc>
nums: Vec<Int, argc>
Yeah, C++ arrays are literally that.
replyArrays are not dynamically sized though (handling runtime sizes) and don’t have efficient append/concat. The point of the dependent types is that you can have the type system track that concat creates an M+N length vector, sort preserves length (and adds a sorted guarantee that slice preserves), etc.
Sure you can do a lot with templates, but that’s advanced templates not just “C++ arrays” in a throwaway “literally that” way.
replydeleted
replydeleted
reply