undefined

points

by vkgcr9 hours ago |

comments

by hoppp9 hours ago|

[-]

It boils down to architecting systems.

Using AI is one element for a pipeline that produces code.

If you architect well what you generate then it stops being slop and starts behaving like a cohesive system where code is produced automatically.

by vkgcr8 hours ago|

parent|

[-]

I don't think there will ever be a way to "steer" an LLM such that it produces the correct output 100% of the time. You can certainly context engineer your way to better outputs, but you can never reduce to 0 the possibility of the LLM making a mistake, due to inherent limitations of the LLM (perhaps what you ask of it wasn't RL'd during its training), but especially due to the fact that it will often misunderstand requirements and make unjustified assumptions.

So, no, you will never be able to produce code automatically. You can and certainly should use AI as a code assistant to write code faster, but the human should always 1) make all important architectural decisions and 2) carefully review the outputted code

The human bottleneck is necessary.

by hoppp8 hours ago|

parent|

[-]

Not correct but you can plan in the mistakes. E.g: I designed a modular monolith where each module is strongly decoupled so an LLM can work on it individually and if it breaks it can be ripped out and replaced without damage to the rest of the system.

So the system accounts for mistakes and has protocols for removing bad modules/code.

If part of the system go down the rest will remain functional

Edit: and yes audits are part of the pipeline

by vkgcr8 hours ago|

parent|

[-]

Sure but that means accepting the cost of having faulty modules running in production (and there is no guarantee that your LLM could fix a faulty module, perhaps it will just replace it with a module with different bugs. There's no way to know!)

This obviously won't fly for "serious" software such as airplane software, but I'm willing to guess that even customers of a low-stakes SaaS would get pissed off if your product regularly turns up new bugs.

I think it's easier to just do code reviews...

by hoppp8 hours ago|

parent|

[-]

There are code reviews. At no point did I write no reviews, that's something you inferred.

Both me and LLMs are constantly auditing software.

In-fact auditing and QA is mainly what I do now and human audits become much easier when it's scoped to modules.

by anal_reactor7 hours ago|

parent|

prev|

[-]

> I don't think there will ever be a way to "steer" an LLM such that it produces the correct output 100% of the time.

You cannot do that with humans either.

by EPWN3D7 hours ago|

parent|

[-]

The difference with humans is that you can incentivize them to try and do the right thing via money. Humans are error-prone and slow, but they don't have an alignment problem, and the types of mistakes they'll make are pretty much understood. There are entire programming languages designed to minimize those types of mistakes.

The types of mistakes LLMs will make are failures of alignment, that is, basically unpredictable. They could fuck up anything. Where a human fucks up by introducing a crash, the LLM fucks up by just not solving the problem you asked it to solve.

It will happen less and less as they get better and better, but as the volume of software output keeps growing, it will happen with a consistent defect rate. And we won't be able to catch it prior to shipping, because humans won't be able to review all the code or verify all the tests.

I don't think anyone's started taking this seriously yet.

by anal_reactor5 hours ago|

parent|

[-]

> The difference with humans is that you can incentivize them to try and do the right thing via money.

Gen Z has learned that usually, your dedication to work is not tied to the monetary reward.

> Where a human fucks up by introducing a crash, the LLM fucks up by just not solving the problem you asked it to solve.

Have you ever done a group work project at school?

by insane_dreamer4 hours ago|

parent|

prev|

[-]