It’s about singular papers with too good to be true results. You can find these in humans too.
The rational mind should not be seeing singular papers and assuming they’re correct. There are a lot of incentives for researchers to publish amazing results that benefit their career. They find ways to publish these through small sample sizes, p-hacking, or worse like faking results.
The amazing results usually disappear in larger studies by more rigorous researchers. There are so many papers showing amazing things in a handful of mice in a lab or even human volunteers that do not appear again in properly powered studies.
I never said we have sufficient evidence to act. But "too good to be true" + "singular paper" together can become an unfalsifiable dismissal - by that logic, every important result looks suspicious before it replicates. The interesting question is what priors should update our confidence here.
Stanford/Arc Institute and published in Nature + mechanistic grounding + prior research on gut-brain axis gives me way more confidence than average, but you're right, that's not nearly enough for most, but quite sufficient for me, and surely others with informed priors or a strong motive.
Every important result should look suspicious before replication. This is the rational way to interpret early research.
You should not allow your mental probability distribution to be anchored around the first claim you see that is proposed as a paper. In the modern publishing environment, a heuristic of assuming singular results will not replicate would be accurate more often than assuming they’re true.
This isn’t intuitively obvious until you’ve read a lot of papers. It’s unfortunate but true.
Even some of the widely accepted findings like the benefits of fish oil supplementation are having a hard time replicating in large scale studies. Go back 10 years and it was almost universally accepted that those early fish oil studies must be true.