The problem isn't fixing it. The problem is detecting it's a problem in the first place.
Stack this with dozens of over "1%" cases and you're in for a world of hurt.
So yes it doesn't matter. But it also does.
This reminds me a bit of security at our company. "Technically I don't need to escape that because it's just an integer." My response always is: technically you don't have to, but technically every dev who reads it needs to check that its an integer. Make the code obviously correct.
You often find issues “in practice” when your users find them. User trust is a limited resource that does not currently scale with LLM code generation rates.
> Tell it not to. Instruct it, shape the process. Define your terms, document your ideal code style. I guarantee you it's better at it than you are once it knows what you want.
I (somewhat, it will choose its own path at odd times) agree, but the rate that it generates code dwarfs mine, so it’s generating issues “in practice” at levels I could never reach. AI + human is going to beat either in isolation in all cases, and I don’t see this ever changing until they can read our minds.
At my company, we have a “10x” AI team that doesn’t read the code. They had a lot of clout until they launched and were reprimanded for a staggering number of issues ranging from technical bugs to weird UX choices.
I’m not sure if their issues were caused by not reading the code or not knowing what they don’t know. Based on personal experience, if I stopped reading the code it would cause similar issues in my app, so I still read the code.
> hear ya'll geniuses shouting in unison our devs suck
The software world is huge. I’ve worked with devs who spent their whole careers in relational dbs and never heard of isolation levels. Entire modes of failure they don’t even know exist. I imagine they’d see code and claim it’s great where others would see the holes. And both could be right: there are business domains where it doesn’t actually matter.
The range of takes on LLMs is downstream of this diversity.