Make an elegant UI that isn't awkward to use without excessive verbosity (mix of coding, design and UX).

Architect a system for long-term maintainability.

Encode the right amount of business logic into tests, without being overly strict.

Models seem quite poor at all of these IME. Essentially any time there are qualitative tradeoffs they are not great at making them.

Right now, I'd probably pick the task of writing a graphical interface, say a webapp, with a bunch of different gestures and having it "feel good" to users. That's a difficult evaluation to automate, at least for now, which is probably why they're not that good at it yet.

They'd prototype and adjust a lot faster than a human coder, though.

How do you define "good coder"? I've yet to see LLMs that write better than what I consider the best human code. I think Michael Hashimoto showed that not too far back where getting LLMs to optimize for a problem resulted in code better than naive but much worse than expert and that's before even talking about how that code looks/reads.

I'll admit I'm not on the very cutting edge models but LLMs make weird tradeoffs and lack creativity. They will sometimes write marginally better but way over-complicated code for no good reason. That can have performance implications and they're generally not "aware". They can gain awareness via profiling but again run into limited creativity or big picture understanding. They are good with cliche but not as good with novel situations. With human guidance this is still productive/useful but as the systems grow more complex they are less able to reason correctly about them.

I would say that in certain situations they outperform the average software developer. They definitely outperform in speed and output. So they're like a maybe 1600 ELO chess player for software. I.e. they play chess better than all non-chess players. They play chess better than many chess players. But they're not "master" or "gm" level across the entire discipline. But they can play 100k chess games in the time it takes me to figure out my move...

I think there’s still value in stuff that people can see, like frontend design. Because even if LLMs are technically better, every site is going to look the same. What’s good very quickly becomes cliché when everyone does it.

I can mediate with project manager about some requirements which should not be implemented. I can make decisions to simplify the system without bothering my manager with endless details. I can create short and concise report.

I can say "no".

I can store passwords in the database unencrypted and unhashed.

So far AI that I've used failed spectacularly on these prospects.

They're like "yes-man". They'll try to do everything you ask of them. Even if you ask nonsense, they'll not push back.

They'll go crazy about security stuff, even not relevant to the current situation.

They'll complicate the code endlessly for situations which will not arise, making the code an incomprehensible mess.

They'll write so many useless words that any reader will forget what this report is about before then reach the middle of it.

They'll never iterate on code until it looks good. They'll stop when test pass. They don't have aesthetical sense of beauty.

They love to cheat too often. And you can't do nothing about that. You can't fine them or fire them, and they know about it.

They're smart but I'm not convinced they're ready to replace human in mundane engineering jobs.