undefined

points

[-]

In my experience you're just replicating the problem. Slop code doesn't stop being slop code when you add in slop tests to account for the sloppiness. If you generate code at a certain velocity, you eventually get to a point where you generate tests the same way you generate code. The model does its best effort in creating tests, but you'll inevitably get to a point in your work where the tests themselves are faulty and unreliable, the same sloppiness you were trying to control for.

There have been multiple times where my software dev team has gone back to prior implemented tests they had the model generate and found that after some time eventually the model got frustrated with the test no longer passing eventually modifying it to fit current needs around a feature push. You can probably anticipate how we came to discover this, as it ended up introducing a regression around multiple edge cases which the test was meant to control for.

The only answer is to inject actual engineering into the process as much as possible without giving it all up to the models to generate more slop in response to slop. tests are a great answer, but only if you treat them with serious intentionality.

by cadamsdotcom3 hours ago|

parent|

[-]

Your engineers weren't doing what I'm describing I promise.

There are multiple places in my post where I explicitly describe the process by which you intimately:

- make the agent do verified work (the e2e tests and test driven thing)

- watch the agent work (the second monitor thing)

- read your own code (the tests/prod mirror thing

- play with the work and perfect it (the tech debt and UI perfect thing)

Plus a ton of other places in my post I very clearly describe being intimately involved in building and selling the work.

That to me is basic god damn competency. What your engineers did is a fad, and it won't last, and I'm sorry they abdicated their professional duty.

If you wish to screen for that sort of shit in future/current hires, show them my post and see if they turn up their noses or if they get excited and creative. You want only the latter.