Still a bit early but I'm working on kiwi, a k-dialect that can lower to Apple MLX.

Currently supports CPU and GPU on macOS and CPU on linux.

https://kiwilang.com

https://github.com/kiwi-array-lang/kiwi

Kiwi runs computations on small dense arrays in its own runtime, when they are larger it will lower to MLX CPU and eventually to MLX GPU when it is worth it.

As user you don't have to change any code, you just write k.

I'm sure there are other languages designed to take advantage of modern GPUs.

But even with SIMD you can get quite far with array oriented code and many array language implementations will make use of it (BQN, ngn/growler/k, goal, ktye k has a version with SIMD support, …)

Both ahead of time compilers and JIT compilers often perform autovectorization of tight loops. The problem is that lots of hot loops are not necessarily simple loops, and in particular a lot of source code is written in a way which uses sequential dependencies that can’t be modeled in SIMD code. Aside from undefined behavior in C/C++, most compilers will fail to autovectorize because doing so would very slightly change the behavior of your code in a very hard to understand way.

Intel Ispc is a compiler for a C superset language that targets CPU SIMD and GPUs.

If you're happy with NumPy's API, then surely JAX is exactly what you're looking for.

>I still don’t understand why we lack a language that will take uncomplicated computation heavy code and turn it into SIMD / multi thread / multiprocessing / GPU code with minimal additional syntax.

It's already (partly) existed called D language, by default it's garbage collected (GC), can also be program without it or hybrid. It's a modern, backward compatible with C and it's included in GCC.

The linear algebra system in D or Mir GLAS is standalone BLAS implementation written directly in D [1]. It's already proven faster than the other widely existing conventional BLAS like OpenBLAS back in 2016, about ten years ago!

This popular OpenBLAS include Fortran based LAPACK (yes you read it right Fortran) and it is being used by almost all data processing languages currently Matlab, Julia, Rust and also Mojo [2].

Interestingly there is a very early stage of standalone BLAS implementation written directly in Mojo namely mojoBLAS similar to Mir GLAS just started very recently [3].

>Surely this is the sort of thing compiler / language design nerds dream about?

You can say this again.

Especially on the GC side of the programming language since this SIMD / multi thread / multiprocessing / GPU can be abstracted away.

Actually someone recently proposed VGC or virtualized garbage collector for Python in C++ for heteregenous GC [4],[5]. However, the current evaluation excludes JIT compilation, AOT optimization, SIMD acceleration, and GPU offloading.

[1] OpenBLAS:

https://en.wikipedia.org/wiki/OpenBLAS

[2] Numeric age for D: Mir GLAS is faster than OpenBLAS and Eigen:

http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...

[3] mojoBLAS:

https://github.com/shivasankarka/mojoBLAS

[4] Virtual Garbage Collector (VGC): A Zone-Based Garbage Collection Architecture for Python's Parallel Runtime:

https://arxiv.org/abs/2512.23768

[5] VGC-for-arxiv:

https://github.com/Abdullahlab-n/VGC-for-arxiv