You might be giving it too much leeway.
Opus and Fable are more trustworthy, but even today I find they very quickly go off the rails for research tasks if I don't give them short chunks of work.
They're not ready for prime time automation of research (and before someone mentions the math results, that required absurd amounts of computation time and has immediate feedback in the form of Lean)
I guess when you ask these things to take on the role of a senior scientist it picks up some natural stubbornness in the training data.
> I would never trust an AI to make a scientific conclusion, but they're very good at some things
> In this case, usually it will screw up and overfit, but it's a good smoke test.
> it's being used for tedious things that provide some value
I see a lot of comments like this about AI use in general, and I suspect there’s a kind of learned helplessness combined with anchoring to earlier experiences. Someone tries a model, encounters a few failures, and forms a fairly permanent view of what these systems can do.
My concern is that people sometimes turn an expectation of failure into a permanent ceiling on how much they’ll even experiment with these tools. Their limited use then keeps confirming their limited expectations.
EDIT: This came off as criticizing the user, but this is entirely to validate my own thesis that for quite a long time nobody has been able to crack the usability challenge when it comes to non-coding knowledge work for these harnesses. It's fascinating because it seems like it's such an easy problem to solve, but in reality it's a complex product problem.
Correct. As of Q4 2026, in my experience they are only trustworthy for surface level use cases in the domain of computational chemistry / in-silico high throughout screening, unless you are working on a highly constrained, immediately verifiable and well-studied problem like protein docking or structure-property relations.
> anchoring to earlier experiences.
I have observed the models improving over time. I am basing my post on things at the level of Opus 5.5 / GPT-6.1 Sol / Fable 5.1
> Their limited use then keeps confirming their limited expectations.
Every time a new model comes out or a harness I use gets a new feature, I do try to push it until it breaks. A swarm of GPT Lunas at max context is highly effective at finding information or condensing highly disparate data sources; that wasn't even a thing a year ago in most harnesses.
But even the smartest models like Fable will get sidetracked, or make bad decisions because they do not have enough context of the problem domain. As of Q4 2026.
I'd put their autonomous intelligence in this particular domain at the level of a well-read first year grad student. Phenomenal at knowing things, but unable to question their own assumptions or invalidate their own work unless you hold their hand in the form of a prompt, skill, or direct instruction.
> unless you hold their hand in the form of a prompt, skill, or direct instruction.
The hand-holding and inability to execute automated tasks is a very well known failure mode.
The requirement for me to prod, redirect and continuously steer is the number one pitfall when I did the Muse Open Kernel challenge using a long-horizon search/improvement swarm (or whatever the term is... just don't call it 'RSI').
Using plateau avoidance techniques such as https://arxiv.org/abs/2610.07447 and building harnesses around these is what helped me.
“Plotting with natural language: "Create a scatter plot of column A ver….”
No, the tedium of boilerplate code was a feature. Charts were generated by template so styling was consistent, making charts a bit more bland but easier to consume and check. You cannot achieve that same level of consistency or predictability with ai without reinventing a wheel. The models change too fast, and they all want to differentiate themselves for brand reasons, and the emphasis is on fast and easy, not on slow and foolproof, there’s no documentation of the work the way code provides documentation and an easy audit trail. You can restrict the ai by giving specs (use ggplot, use color template foo, create readable code with unit tests), but these are all similar tradeoffs as you might make with human labor too.
What llm do is to simply lower human value, not remove it entirely.
Just as results needed checking and replication before llm, they will continue to need checking and replication by humans. The humans won’t be compensated nearly as much, but humans are humans and so are not.
This is why reading the whole post is important :)
You seem to think I'm suggesting the use an image generator for plotting. Don't do that, it will screw that up, image models are not made for that. The use for AI here is to save 5 minutes writing the code to generate the plot, or to edit the style (again via code) of the plot.
If you do plt.scatter(x,y, c="blue", marker="o") that is a function call in Matplotlib. Its API notoriously sucks. Most Python plotting libraries just create convenience functions around this library because although the API (actually there are two and the documentation is not consistent on which to use) is hot garbage, and the documentation is labyrinthine, the plotting engine itself is powerful.
> What llm do is to simply lower human value, not remove it entirely.
The value in a plot is not the exact code used to make it (beyond the code being correct). The value is in the visualization giving the reader an "ah-ha" moment. I would much rather take the 5 minutes I save writing a function call, and use it to spend more time examining the plot itself.
This is something the LLMs are really good for, at least for compiler papers.