I still think this is because, on a new release, it works on some prompts the previous ones did badly at, because new weights do well on a different set of prompts.

Then after a few days you notice the prompts that it does badly on that the old ones did fine with and everyone is convinced there's a regression when it's just a different part of prompt space

I find Astra to be weirdly stupid in the sense you have to force it to spend time on something (fix this architectural issue and refactor), then it’s stupidly smart.

It prioritises getting something working over making something good during the 1-shot phase and outputs maximum slop.