I would never trust an AI to make a scientific conclusion, but they're very good at some things:
- Plotting with natural language: "Create a scatter plot of column A versus column B, log scale Y and use stars sized by column C." Every plotting library (except maybe ggplot2 / plotnind) sucks, and the boilerplate never gets less tedius.
- Literature search, e.g. "Have there been any peer reviewed papers where they applied X method to Y domain"
- Complex visualization, e.g. "Make a self contained HTML file with a 3D interactive visualization of this graph. Add ball and spring position optimization and let me move nodes"
- Trying random stuff that would normally be too time consuming to be worth it, eg "Here is a dataset. Try regression methods A, B, and C and report a table." In this case, usually it will screw up and overfit, but it's a good smoke test.
Note how in all of these, it's being used for tedious things that provide some value, but ultimately they are only providing information to make a decision on.
Granted, I would like to have a research assistant I could train, but if your company will not create a position then there is not much you can do.
You might be giving it too much leeway.
Opus and Fable are more trustworthy, but even today I find they very quickly go off the rails for research tasks if I don't give them short chunks of work.
They're not ready for prime time automation of research (and before someone mentions the math results, that required absurd amounts of computation time and has immediate feedback in the form of Lean)
I guess when you ask these things to take on the role of a senior scientist it picks up some natural stubbornness in the training data.
> I would never trust an AI to make a scientific conclusion, but they're very good at some things
> In this case, usually it will screw up and overfit, but it's a good smoke test.
> it's being used for tedious things that provide some value
I see a lot of comments like this about AI use in general, and I suspect there’s a kind of learned helplessness combined with anchoring to earlier experiences. Someone tries a model, encounters a few failures, and forms a fairly permanent view of what these systems can do.
My concern is that people sometimes turn an expectation of failure into a permanent ceiling on how much they’ll even experiment with these tools. Their limited use then keeps confirming their limited expectations.
EDIT: This came off as criticizing the user, but this is entirely to validate my own thesis that for quite a long time nobody has been able to crack the usability challenge when it comes to non-coding knowledge work for these harnesses. It's fascinating because it seems like it's such an easy problem to solve, but in reality it's a complex product problem.
Correct. As of Q4 2026, in my experience they are only trustworthy for surface level use cases in the domain of computational chemistry / in-silico high throughout screening, unless you are working on a highly constrained, immediately verifiable and well-studied problem like protein docking or structure-property relations.
> anchoring to earlier experiences.
I have observed the models improving over time. I am basing my post on things at the level of Opus 5.5 / GPT-6.1 Sol / Fable 5.1
> Their limited use then keeps confirming their limited expectations.
Every time a new model comes out or a harness I use gets a new feature, I do try to push it until it breaks. A swarm of GPT Lunas at max context is highly effective at finding information or condensing highly disparate data sources; that wasn't even a thing a year ago in most harnesses.
But even the smartest models like Fable will get sidetracked, or make bad decisions because they do not have enough context of the problem domain. As of Q4 2026.
I'd put their autonomous intelligence in this particular domain at the level of a well-read first year grad student. Phenomenal at knowing things, but unable to question their own assumptions or invalidate their own work unless you hold their hand in the form of a prompt, skill, or direct instruction.
> unless you hold their hand in the form of a prompt, skill, or direct instruction.
The hand-holding and inability to execute automated tasks is a very well known failure mode.
The requirement for me to prod, redirect and continuously steer is the number one pitfall when I did the Muse Open Kernel challenge using a long-horizon search/improvement swarm (or whatever the term is... just don't call it 'RSI').
Using plateau avoidance techniques such as https://arxiv.org/abs/2610.07447 and building harnesses around these is what helped me.
This is something the LLMs are really good for, at least for compiler papers.
As a hobby I run https://smalldocs.org / https://github.com/espressoplease/smalldocs an agent agnositc artifact layer, so I'm looking for feedback on artifact use in general.
Here are some sample scientific SmallDoc artifacts:
Palmer penguins: body mass, flipper length and species - https://smalldocs.org/s/YrJDCfCD8-6j_uRE0pNCbI#k=odJRxWOIySV...
AlphaFold 3: capabilities, evidence and operational limits - https://smalldocs.org/s/rRRuByQhmhZNBH_4RNCx1t#k=S0Jhyf72Q6M...
The missing feature for docs is now GDrive, ie organizing and cataloging your artifacts to build a coherent long-lived KB. But for that you can still export.
Anthropic added a while back the ability to let clients provide their own API key, again I think that’s under-explored and will become more relevant as AI gets more penetration.
Building out quick SPAs, HTML pages, etc is a great use, I think the innovation will be making these things supportable and usable for more than one-shot single-user artifacts. Eg user login, analytics, etc.
Did they get permission to use researchers' data or did they monitor and extract data without consent? Considering the kerfuffle OpenAI generated with their math proofs, you'd think Anthropic would take a hint and be clear that they did the research with consent, though that assumes they got consent.
Seriously, all the industry researchers at these companies went through IRB training during their PhD; have they just conveniently forgotten?