So far the trend is pointing to AI being better at finding problems than designing software to avoid them.
replyWhich I'm fine with really, but it means you need to be very careful letting it iterate on code until it finds no problems.
If anyone here hasn't experienced agents going into a loop finding "problems" and "fixing" them by adding more brittle complexity which introduces 10 more problems I'd be surprised. It's like the biggest experience of working with agents RN
replyThe worst part is being thorough takes so much more work - they often do raise valid problems but you just can't set them in a loop because of this tendancy so you end up with way more work and hard questions
It'll never scew to far of as long as defence is a RL target. Better red team just means the blue team has stronger target environmenta to RL for.
reply