upvote
That's a fair clarification.

I never said we have sufficient evidence to act. But "too good to be true" + "singular paper" together can become an unfalsifiable dismissal - by that logic, every important result looks suspicious before it replicates. The interesting question is what priors should update our confidence here.

Stanford/Arc Institute and published in Nature + mechanistic grounding + prior research on gut-brain axis gives me way more confidence than average, but you're right, that's not nearly enough for most, but quite sufficient for me, and surely others with informed priors or a strong motive.

reply
> by that logic, every important result looks suspicious before it replicates

Every important result should look suspicious before replication. This is the rational way to interpret early research.

You should not allow your mental probability distribution to be anchored around the first claim you see that is proposed as a paper. In the modern publishing environment, a heuristic of assuming singular results will not replicate would be accurate more often than assuming they’re true.

This isn’t intuitively obvious until you’ve read a lot of papers. It’s unfortunate but true.

Even some of the widely accepted findings like the benefits of fish oil supplementation are having a hard time replicating in large scale studies. Go back 10 years and it was almost universally accepted that those early fish oil studies must be true.

reply