upvote
It is interesting how all these famous non-replicating psych paper results feel reasonably intuitive to me. Like I can buy that having a more structured and detailed deadline helps you procrastinate less. I can even buy that doing a goofy little "power pose" exercise kind of hypes yourself up to give some self confidence and do better on an interview or whatever.

You would almost expect the opposite and that all the famous studies would be the ones that run counter to intuition. Maybe those get more replication attempts, or people feel less confident faking them.

The direction is intuitive, but the effect sizes are, as you say, unusually large. I'm no statistician, but if you have an effect size that's several multiples of the standard deviation, that would mean that it dominates the natural variation in results right? So you're kind of making an implicit claim that other factors in play (for eg in this proofreading task, how busy someone is or their reading level) are peanuts in comparison.

Further, if there are these relatively simple factors like how a deadline is expressed that make you several times better, you'd expect anybody who stumbles into several of them and stacks them together to be absolutely off the charts. Somebody who listens to Mozart, gets the right priming, doesn't deplete their ego, and receives short deadlines is suddenly like 50X more productive, which doesn't seem to match reality.

reply