Reversals in psychology

On collating the replication crisis in psychology, and announcing a new crowdsourced replication effort.

Like Comment
Read the paper

Like anyone curious about humans, I read a lot of psychological findings. The last decade saw the rise of behavioural economics and many influential popular books about theories in social psychology. I read all the Daniels: Kahneman, Ariely, Gilbert, Schachter.

The decade also had us acknowledge the replication crisis throughout the sciences. I learned about each fresh disappointment from Andrew Gelman's blog, and from extremely-online psychologists. But I didn't find anyone collecting these results - and, 10 years into the crisis, my friends still repeated the old claims without qualification: claims of depleted egos, of powerful poses, of large unconscious influences on all kinds of high-level behaviour. So I did it.

I called my list "reversals in psychology" by analogy with medical reversals, the evidence-based removal of a treatment from clinical practice.

Note that I am no expert; my field is artificial intelligence, a world away from the complexities of human experiments or the latent and reflexive phenomena of social science. Fortunately my method was simple: look up the original study's effect size, and compare to the best multi-study replication effect. 

The post is just a collection of other people's hard work (although it was fun to reanalyse the original power-pose data). As such, the response was outsized; kind messages from Jonathan Baron, Andrew Gelman, James Heathers, and hundreds of other psychologists working to improve their field. 

It's not a hit piece. In some ways the replication crisis is a good thing: it shows that psychology has some self-correction. And the prominence of psychology among all the replication crises is perhaps a testament to the relative openness of data and code in the field: it is more possible to replicate and reanalyse.

The open science organisation FORRT are currently turning the post into a validated crowdsourced effort. I encourage anyone to check our working, to report reversals which are themselves now reversed, and to help the list reflect the shifting, messy bodies of current evidence.

New project, seeking collaborators: forrt.org/reversals/

Gavin Leech

PhD Candidate in AI, University of Bristol