IMHO, Microsoft made the correct approach on .NET.
For LLMs, I prefer C# and C++ instead of TypeScript, JavaScript or Python as the static + compiled language factor keeps the coding agents on track. Plus, they have a true threading/async implementation.
The actual physical RAM is still entirely available to other applications. It's just made the OS know it might want that many pages. Until there's data in the pages, they will not count towards total RSS.
It's the kind of things some sysadmins used to gripe to me about and I would question whether they should be in charge of a machine at all.
To repeat: just because an application mmaps a large region doesn't mean the OS has actually given it all that physical RAM. It's merely made sure the pagetable knows about it.
If the program is actively using that allocation, that's fine. My problem is with the runtime hoarding RAM when it should have been freed after GC back to the OS.
Then there's also the JVM not handling peaks well because it hit the max heap size, while you still could rely on the OS doing its job to shuffle stuff to swap temporarily. I still see JVM OOMs in my $dayjob's product while the OS has plenty of free physical memory. It is stupid.
I mean, we have malloc() and free(), they are in the stdlib for a reason :)
The JVM seems to follow a philosophy where it assumes it is the only process running besides PID 1, which is valid for some scenarios, but not for others.