LLMs emitting code are a great workaround for "bury the complexity, I just need the result" and terrible for "this codebase produces long-term value". People are slow to learn the difference, and the AI gold rush profits from the confusion. We thought television would bring Shakespeare to the masses. It didn't, but theater was still there for Patrick Stewart (RIP). Software engineering will be okay.
I'm not at all convinced that LLMs will kill software engineering but I'm equally unconvinced that it will be an overall positive for innovation in the field, at least while we're basically all running things through the same handful of closed model orgs. It's just a really weird time where I can't decide if I'd rather jump back and work in the world of five years ago or take my chances and jump forward the same period.
And yeah, I’m not sure how that goes away either - life is full of tradeoffs, how do you RL things with no objective truth?
There's so much to learn in the world. The domain knowledge I used to carry about software, I hope I forget most of it. I hope I learn things that let me do things I never could've conceived of before.
It was pretty good, understood what I wanted to do, and broke it up in playbooks/roles/etc. quite well. But then it did things like continuously overwrite the LetsEncrypt cert for each domain with the next one in a loop. It understood "this needs to be a loop" and how to optimize that loop, but not that it was going to overwrite each domain's key with the next.
I'm a pretty good sysadmin, so I recognized this and could take what it did and fix it, but it required a deeper understanding of the underside of nginx/LetsEncrypt and the filesystem to even see what it was doing.
Of course, once you run the playbook, test a domain and see its SSL cert is all jacked up you'll go looking for WTF happened, but just blindly taking the output as given meant it was horribly broken.
It may be that Ansible, by its nature, is a bit sui generis because it touches so many unconnected domains, and therefore is harder than outputting Python. I don't know. But one needs more than a surface level understanding of those domains if you want to catch the LLM being stupid.