upvote
Yeah, about a year ago the labs figured out that effective intelligence is a function of persistence as much as anything else. So the models started getting scary persistent late last year, and the trend has continued. There was another jump a few months ago.
reply
they realized the paperclip maximizer and the grey goo are _not the torment nexus_, but reliable patterns of discovery!
reply
I believe this is part of the complaints of new models taking longer/requiring higher spend - they go the extra mile on verification, regardless of whether their change is correct already or not. So on problems that an earlier model one-shotted an answer to and did some lighter verification, the newer models might take longer to come back to the user due to running all the tests for your software they could find.
reply
I noticed that too. I'm thinking about adding a prompt to disable those tests. We have the unit and feature tests anyway: add to them. I'm OK with the syntax checks: I work with interpreted languages, Ruby, JS, Python.
reply
Well, it seems that Linus doesn't use those:

> And this was a debug session from hell, enormously helped by an AI doing much of the grunt-work.

> I'd like to call it my tireless helper, but the AI several times stated flat out that this was impossible and unsolvable and that we should just write a report about it.

> I suspect those things have been trained by people who may not be quite as stubborn as I am.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

reply
Both things can be true. I’ve noticed both the same thing the parent posted and what Linus posted and my vibe on the split (I haven’t kept detailed notes) is that on greenfield code they tend to maybe over-verify and on brownfield code or data analysis they sometimes give up too early or… I’m not sure, need a bit of encouragement to keep pulling at threads.

On the data analysis side, something specific I’ve noticed is an (understandable) bias towards computing numerical statistics, which they do very well and reading the post-analysis report has significantly improved my own “statistical thinking” approach overall. Numerical statistics are cool and understandably what a text-based LLM is going to want to work with, but asking the model to produce time-domain and frequency-domain plots of, say, specific events has multiple times resulted in “trying to plot this out has shown the opposite of what I concluded numerically… recalculating…” There’s still a pretty significant review and critically assess step for me, especially since the actions I take as a result of the analysis are pretty expensive, especially if they steer the next data collection run in a useless or harmful direction.

reply
deleted
reply
Any idea how being persistent is trained? I've noticed that telling an LLM that it needs to think some more sometimes produces better results, but the claim here is that "they are very persistent" and "...kept going...".
reply
It's from work like this:

https://arxiv.org/abs/2309.11495

A RL pipeline can reinforce verification behaviour even better than simple prompting.

reply
Your explanation makes sense, but I don’t know how you can make a sweeping generalization about more recently released models without qualifying it.

This is says more about humans tendency to pattern match than anything else.

X works better than Y only is only a useful observation if we are using the same X and Y in a similar context, with similar parameters. Kind of goes out the window without it and I think this is part of why people have such vastly different opinions about the same technologies. We’re all talking past each other.

reply