upvote
Replication is unpaid QA on someone else's product. Nobody gets tenure for finding the bug, hence the field has been meaning to get around to it for 20 years.
reply
Universities, institutions and publications who care about their reputation should pay people to replicate studies performed by people they employ in their name. Otherwise they will loose their trustworthiness.

But perhaps it is not as big a shame for them as some think it should be...

reply
And the incentives are arguably worse than QA: you get little credit if the result replicates, and potentially a lot of grief if it doesn't. Novel results build careers. Checking them mostly doesn't
reply
On the other hand, this seems like a super interesting path for an (unglamorous) PhD -- replicate the N most important findings in a field, subject to some restrictions (maybe focus on the ones that are "easy", or haven't been replicated yet, or the ones that could be patched together to make an acceptable PhD thesis). This sort of stuff really appeals to me, I think I might end up doing something like that in 5-10 years.
reply
> The study seems extremely easy to replicate (no fancy equipment needed, just ask some people to proofread), and yet nobody did until now

They were just putting it off.

reply
Why do we even accept anything as true without replication by default?
reply
Why do people on this site love nutrition and self-medication fads?

People are surprisingly thoughtless.

reply
Or as one might say, predictably irrational.
reply
Because “science” doesn’t know everything, and people don’t want to wait until their options are scientifically-validated.

Yoga, breathing exercises, meditation and the like were once considered fads and derided by your ‘science’.

reply
> Yoga, breathing exercises, meditation and the like were once considered fads and derided by your ‘science’.

What do you mean "once"?

reply
Are you a vendor of orgone accumulators or a staunch believer and customer?
reply
How do you decide whether something works or not?
reply
You don't. It is too costly and thus risky. Instead you do a charade and convince authorities (who just want to cover their asses citing some vetted studies) and people that your stuff work.

It is "ass covering" all the way. Not "does it work" all the way...

reply
"easy to replicate (no fancy equipment needed, just ask some people to proofread)"

? This was a human subject study, that would cost $$ and there would be very little upside to confirmation and only a little notoriety with a failure to replicate (which by the way is not dispositive, people might need multiple failed replications to know who to believe, depending on the size of the replication). Maybe you didn't read the post: this was data fraud discovered by statistical irregularities. That is indeed becoming easy to spot with more widespread statistical literacy, at least for unsophisticated jobs like this from decades ago.

reply
> Is there some way to change incentives to get people to try and replicate?

Sure. Pay them to do it.

reply
If the culture shifted such that people were skeptical of original results but excited about replication, replication would be the big news and people would want to do it. How to get there, I don't know.
reply
I seem to remember a lot of hype about room-temperature superconductors, that croaked upon [lack of] replication. That wasn't too long ago.

Before that, cold fusion.

We still have the ion drive that doesn't require fuel. I don't think it has ever been successfully replicated, but I think that it also hasn't been successfully repudiated, either.

This kind of thing has been going on for a long time. I suspect that not all of it is deliberate fraud, but rather, people wanting their preconceived bias to be satisfied.

reply
Replicating superconductor experiment gives you the conductor and the skill.

Both of them are useful as is.

Replicating a social experiment, when successful, don't give you anything new.

reply
No, that would never happen. People would just be skeptical of most findings until they see replications (where we are headed in the social sciences, anyway). If people see replications the original discoverer will get all the accolades, not the (several) replications.
reply
Or maybe we shouldnt call anything "science" until it assumes replication of foundational results.

Physics solved this quite some time ago.

reply
Then it "replication" becomes a target and thus would be gamed instead.

This is a constant tug of war..

reply
[dead]
reply