So it would really be beneficial for the LLMs to be able to have their own environments to write and run code, but also to store those programs for later use automatically.
A lot of our manufacturing ability improvement during the last 200 years, as well as our engineering ability improvement during the last 500 years, is not just the raw power we unlocked, but that we made it deterministic, reliable and reproducible and hence automatable and automatic.
Can you build a new hospital? Up to the local government to decide. How much is your property tax? Only a tax assessor can tell you. Which trees get trimmed and which are left intact? Decided on case by case basis with no strict rules. Which convenience stores are getting audited for tax and food safety compliance this week? There's a middle rank officer who makes that decision every week, my mum used to be one. How high is the bail set in criminal proceedings? Judges are famously very inconsistent on that and it's a good thing. In countries without constitutional gun rights, it's common that the final step in the process of obtaining gun permit is some government employee having a good gut feeling about you, and if they get bad feeling you get denied. And so on and so forth everywhere you look.
Manufacturing is a different beast entirely. I'm not saying we should put Siri and Cortana in charge of bolting down Boeing fuselage. But for most office and bureaucratic tasks, demanding perfect determinism is not only unnecessary but actively harmful.
Nondeterminism is actually a bug in many cases, and training, process and procedure, is how we mitigate it in humans.
Defining every single step of every single task and making them perfectly repeatable works in an assembly line. But as an office assistant - and doubly so as a personal assistant - it creates inefficiency, makes dealing with edge cases very painful, and greatly increases maintenance burden of keeping all procedures up to date. Instead, giving people some leeway in how they approach their tasks and deliberately underspecifying processes to let them exercise their judgement is shown to work much better. Surely you had it covered in your training materials?
Computers can do that!
That said, my point is that determinism is incidental to how current generation of compression algorithms work, not an explicitly desired quality. And even that isn't strictly true, because e.g. x264 in certain multithreaded configurations already produces non-deterministic results that look as good as single-threaded encoding. For this use case - and thousands of others - losing determinism when switching to AI-based solutions really isn't a big deal.
The LLMs empower for building, the determinism improves output-expectation alignment. It’s fundamentally an on device software factory, hypervisor, and storage system.
Remember when everyone was talking about treating computers as cattle rather than pets?
This seems like a weird inversion. Programs as pets instead of cattle. Toxoplasmosis, ahoy...
Think of all those bad sci-fi movies where aliens take over Earth, enslave all humans and treat them like cattle. Very competent cattle that's put in charge of local governments, scientific research etc. but very much still cattle where individuals are entirely interchangeable and killing them is just everyday work. This is me and AI assistants. (I'm the aliens.)
Pros and cons to code being cheap to generate now, it’s always about tradeoffs. I don’t want to build cars, I want to drive cars.
People suck at "good enough". All major engineering disasters were something "good enough" for somebody. People burn down their houses with "good enough" electrical work.