The "we don't read/write assembly anymore" argument doesn't fit though. Given an input x we know without a doubt we'll get output y. With an LLM, that isn't the case no matter how many guard rails, specs, markdown files, diagrams you give it.
Backend code I'm fairly comfortable if the tests are good that it works correctly. It's a much more constrained environment. Obviously there can be things like performance issues that creep in or problems when it mocks too much of the environment.
Meanwhile frontend code I have to consider different browsers, screen sizes, a11y issues, and so on, which agents can certainly do but you have to be fairly explicit about it, and they don't always make the best tradeoffs.
Frontend code is probably far harder to apply this technique to.
Sweet, we could even develop a standard language(s) to lay out such specs, and write in specialised text editors that would help with debugging. Hell we could even call it ‘programming’
While you're being tongue in cheek, what you're missing is that the spec you'd be working on in some cases can be compressed to a small fraction the size of the implementation code. Alternatively, it can be written for clarity and correctness rather than performance, and then the implementation can be optimized like crazy without ever worrying about correctness.
I think the problem in this case was: 1. Unusual constraints, that kept nudging the model to overdesign or misunderstand parts of the solution at every turn. 2. One code-comment it found and thankfully mentioned, that a previous session had made up and left there as a fabricated requirement. The model trusted this "requirement" more than my instructions, which unfortunately made sense, since it was indeed written and part of the code.
I'm working on a system that has a mix of trust boundaries with a few perspectives that are a-typical and it trips up the models working on it. Fable 5.2 didn't do better at all.
I have tried to put clear instructions in an agent architecture file, but I haven't managed to get it fluent yet.
>Only exception might be the upper echelons of optimization, like language authors or compiler level work. But even then…
Optimization work is especially suited for LLMs. They're great at hill-climbing problems and have infinite patience to test different ideas.