The replication crisis in psychology is the finding that many published results do not hold up when other labs repeat them: in the 2015 Reproducibility Project, 97% of 100 original studies were statistically significant but only 36% of the replications were, and effect sizes roughly halved. Ego depletion, the pen-in-mouth facial-feedback effect and the hormonal claims of power posing failed large preregistered replications. Many other effects survived, usually smaller than first reported, and in 2026 the SCORE project found that about half of social and behavioural science findings replicate.
- 36% of 100 replications were significant in 2015, against 97% of the originals.
- d = 0.60 → 0.15: the median effect size fell by three-quarters in Many Labs 2.
- Ego depletion failed in 23 labs and again in 36 labs, with a protocol its own supporters chose.
- Power posing: a co-author of the 2010 paper said in 2016 she no longer believes the effect is real.
- About half of 274 social and behavioural findings replicated in the SCORE project (Nature, 2026).
If you have read a self-help book in the last fifteen years, you have met findings that were later tested at scale: willpower as a muscle that tires, two minutes of a power pose before an interview, smiling to feel happier.
This post is a scorecard. For each claim it gives the original idea, the size of the replication, and what it found, so you can sort the advice you have absorbed into three bins:
- Holds: replicated at roughly the published size;
- Holds but smaller: real, but a fraction of the original effect;
- Does not hold: indistinguishable from zero in large, preregistered tests.
What is the replication crisis in psychology?
The replication crisis is the gap between how reliable published psychology looked and how often its findings reappear when independent teams repeat the method with large samples. The defining study is the Open Science Collaboration's "Estimating the reproducibility of psychological science", published in Science in August 2015.
The team repeated 100 experimental and correlational studies from three leading psychology journals, using high-powered designs and the original materials where available. The results:
See the data as a table
| Share of studies (%) | |
|---|---|
| Originals with p < .05 | 97 |
| Replications with p < .05 | 36 |
| Original effect inside replication 95% CI | 47 |
| Rated as replicated by the teams | 39 |
| Significant after combining original and replication | 68 |
Source: Open Science Collaboration, Science 2015 · Replication effect sizes were on average about half the original effect sizes. The 68% figure assumes no bias in the original results.
The authors found that replication success was better predicted by the strength of the original evidence (smaller p-values, larger effects) than by who ran either study. By field, cognitive psychology replicated at about 50% and social psychology at about 25%, a split widely cited from the same project.
Replicated 100 studies from three psychology journals with high-powered designs. 97% of originals but 36% of replications were significant; replication effects were about half as large; 47% of original effect sizes fell inside the replication's 95% confidence interval; 39% were judged replicated. Original evidence strength predicted success better than team characteristics. Limit: one attempt per study, and some protocols differed from the originals, so a single failure is evidence of weakness, not proof an effect is zero.
Read the paperWhy do psychology findings fail to replicate?
Mostly because original studies were small and the publishing system rewarded significant, surprising results. With small samples, only studies that happen to overestimate an effect cross p < .05, so the literature fills with inflated numbers.
Researchers also had freedom in which measures, exclusions and analyses to report, often called researcher degrees of freedom, which raises false positives without any fraud. The clearest first-hand account of how that works came from inside one of the most famous studies of the decade.
In 2010, Dana Carney, Amy Cuddy and Andy Yap reported that holding an expansive "power pose" for a couple of minutes raised testosterone, lowered cortisol and increased risk-taking. In 2016, after failed replications, Carney posted a statement on her Berkeley faculty page saying plainly that she did not believe power pose effects were real.
Then she listed what had happened in her lab. The final sample was 42 people after 5 exclusions. Participants were run in chunks while the team checked the result along the way. For risk-taking, a Pearson chi-square gave p = .052 and a likelihood-ratio test gave .05, and the smaller was reported. Several power questions were asked and the ones that worked were kept. The data were real, she wrote; the sample was tiny, and at the time none of it felt like p-hacking.
Dana Carney, UC Berkeley Haas: My position on "Power Poses" (2016)Many Labs 2 shows what happens when you remove those freedoms. Klein and colleagues preregistered replications of 28 findings, each run in about half of 125 samples, for 15,305 participants in 36 countries and territories (Advances in Methods and Practices in Psychological Science, 2018). Fourteen (50%) replicated at a strict p < .0001. Twenty-one of 28 (75%) were smaller than the original, and 9 (32%) went in the opposite direction. The median effect fell from d = 0.60 to d = 0.15.
The most useful finding was about context. Lab versus online administration, task order, and WEIRD versus less-WEIRD samples explained little of the variation. Effects that were real tended to appear in most samples; effects that were not real failed almost everywhere. "It works in the US but not in India" is rarely the explanation for a failed replication.
Preregistered, peer-reviewed protocols for 28 classic and contemporary findings, each run in about half of 125 samples (15,305 participants, 36 countries and territories). 54% replicated at p < .05 and 50% at p < .0001; median d dropped from 0.60 to 0.15; heterogeneity across sites was small and mostly found in the largest effects. Limit: critics argue the samples were still mostly WEIRD and that culture was coded too crudely to detect real cultural moderation.
Read the paper (free)Is ego depletion real?
Not in the form the self-help books describe. Two of the largest replications in psychology found effects indistinguishable from zero. Ego depletion is the idea that self-control draws on a limited resource, so resisting a cookie leaves you less able to persist on a hard puzzle afterwards; earlier meta-analyses had reported a medium-sized effect.
Two tests, two nulls
A Registered Replication Report led by Martin Hagger (Perspectives on Psychological Science, 2016) ran one standardised protocol in 23 labs with 2,141 participants. The pooled effect was d = 0.04, 95% CI −0.07 to 0.15.
Supporters objected that the depleting task was too weak, so Kathleen Vohs, one of the theory's proponents, led a second test: 36 labs, 3,531 participants, two different depletion procedures chosen by experts (Psychological Science, 2021). The confirmatory effect was d = 0.06, not significant, and Bayesian analysis found the data about four times more likely under no effect than under a modest one.
An exploratory result, larger depletion in people who reported more fatigue, is a lead for future work, not a rescue.
Ego depletion failed twice: once with an independent protocol, and once with a protocol its own supporters designed.
36 labs and 3,531 participants ran one of two expert-chosen self-control manipulations followed by a self-control task. The confirmatory effect was d = 0.06 (non-significant); with an informed prior of δ = 0.30, the data were about four times more likely under the null. Ignoring exclusions gave d = 0.08, still weak evidence either way. Limit: tests a paradigm, not every possible version of the resource idea, and the fatigue moderator was exploratory.
Read the paperWhich famous findings failed, and which survived?
The pattern is consistent: dramatic effects from small studies shrank or vanished; subtler effects measured in larger samples mostly held. The scorecard below uses only large, preregistered or multi-lab tests.
| Claim | Original | Replication | Size of test | Result | Verdict |
|---|---|---|---|---|---|
| Ego depletion | Baumeister lab, 1998 onwards | Hagger et al. 2016 | 23 labs, n = 2,141 | d = 0.04 [−0.07, 0.15] | Failed |
| Ego depletion, expert protocol | As above | Vohs et al. 2021 | 36 labs, n = 3,531 | d = 0.06, null favoured 4:1 | Failed |
| Pen in mouth makes cartoons funnier | Strack, Martin & Stepper 1988: 0.82 points on a 10-point scale | Registered Replication Report 2016 | 17 labs | 0.03 points [−0.11, 0.16] | Failed |
| Smiling makes you happier (mimicry, voluntary smile) | Facial feedback hypothesis | Many Smiles Collaboration 2022 | 19 countries, n = 3,878 | Amplified and initiated happiness; pen task inconclusive | Survived, small |
| Power posing raises testosterone, lowers cortisol, increases risk-taking | Carney, Cuddy & Yap 2010 | Ranehill et al. 2015 | n = 200 | No hormone or risk effect; people felt more powerful | Failed (feelings only) |
| Marshmallow test predicts later achievement | Shoda, Mischel & Peake 1990 | Watts, Duncan & Quan 2018 | Large US cohort | Half the original correlation; two-thirds smaller with family controls | Survived, much weaker |
| Flag priming raises conservatism; money priming raises system justification | Social priming studies | Many Labs 1, 2014 | 36 samples, n = 6,344 | Did not replicate | Failed |
| Ten of 13 classic effects in Many Labs 1 | Various | Many Labs 1, 2014 | 36 samples, n = 6,344 | Replicated consistently | Survived |
The facial-feedback story shows why "failed" needs care. The specific pen-in-mouth version failed in 17 labs, but a larger 2022 test of the underlying idea, led by Nicholas Coles across 19 countries, found that mimicking smiles and smiling on purpose did raise happiness, by about as much as looking at mildly pleasant pictures. The theory survived; the famous demonstration did not.

How big is the shrinkage, in numbers?
Across projects, replications find effects roughly half to a quarter of the original size, and replication rates between about a third and three-quarters, depending on the field and the criterion. The chart lines up the headline rates.
See the data as a table
| Replicated (%) | |
|---|---|
| Many Labs 1 (13 effects, 2014) | 77 |
| Many Labs 2, p < .05 (28 effects, 2018) | 54 |
| Many Labs 2, p < .0001 | 50 |
| OSC 2015, cognitive psychology | 50 |
| OSC 2015, all 100 studies | 36 |
| OSC 2015, social psychology | 25 |
Source: Klein et al. 2014 and 2018; Open Science Collaboration 2015 · Many Labs 1: 10 of 13 effects replicated consistently. Criteria differ between projects, so compare within a project, not across.
Use the calculator to see why small original studies are the core of the problem. If the true effect is smaller than a study was designed for, its power falls sharply, and the published "hits" from such studies are the ones that overestimated the effect.
How many participants does a two-group study need?
Did psychology fix the problem?
Partly. Four reforms are now common:
- Preregistration: the hypothesis and analysis are filed before data collection;
- Registered reports: journals accept a study on its method, before results exist;
- Open data and materials, so others can re-analyse and repeat;
- Multi-lab tests that pool thousands of participants across countries.
The 2026 SCORE results suggest that openness tracks reliability. SCORE (Systematizing Confidence in Open Research and Evidence), funded by DARPA and run with the Center for Open Science, published its main results in Nature in April 2026. Over 800 researchers assessed nearly 4,000 claims from social and behavioural science papers published between 2009 and 2018.
In the replication arm, 292 researchers from 199 institutions in 31 countries re-tested 274 findings from 164 papers. Roughly half replicated, with estimates from 29% to 75% depending on the statistical criterion; replicated effects were on average less than half the original size, and smaller than the original in about 70% of cases. Aarhus University's summary of the work stresses the value of open and transparent research, and the authors caution that a failed replication does not by itself prove the original wrong.
The deeper fix is conceptual. Some researchers, notably Christopher Bryan, Elizabeth Tipton and David Yeager, argue that many effects are real but heterogeneous, large for some people and settings and absent for others, so a single average from one sample was never going to replicate cleanly. That argument cuts both ways: it rescues some theories, but it also means the confident one-line advice built on them was never justified.

How should you read a psychology claim now?
Here is the checklist I would apply to any finding before acting on it, built from what predicted replication in these projects:
- Ask for the sample size. A two-group study with 20 to 40 per group can only reliably detect large effects. Most real effects in Many Labs 2 were small.
- Look for a multi-lab or preregistered test. If a claim has only ever been shown by one lab, treat it as a hypothesis.
- Prefer strong original evidence. In OSC 2015, smaller original p-values and larger effects predicted replication.
- Expect shrinkage. Halve the published effect before deciding whether it matters to you.
- Separate the theory from the demonstration. Facial feedback survived even though the pen task failed; ego depletion failed even with the theory's authors designing the test.
The same habits apply to research on mental techniques. The breathing and meditation literature is full of small, unblinded studies; see how they hold up in slow breathing and the vagus nerve and mindfulness hype vs evidence. Bias research has its own replication record in cognitive biases ranked by evidence, and habit advice is sorted the same way in habit techniques ranked by evidence. For the everyday Indian "why" questions answered with research, see Kyun.
What's new
- SCORE: 274 findings from 164 papers re-tested; about half replicate, effects shrink by more than half Aarhus University (Nature study), Jun 2026
- Landmark study finds about half of published social science replicates Constructor University, Apr 2026
- Investigating the replicability of the social and behavioral sciences Nature, 2026
Questions people ask
Is the ego depletion effect of self-control reliable?
No. Two preregistered multi-lab tests, 23 labs (n = 2,141, d = 0.04) and 36 labs (n = 3,531, d = 0.06), found effects consistent with zero. Fatigue may still affect performance, but the idea that a few minutes of resisting temptation reliably drains willpower did not hold up.
How do we know there is a replication crisis?
Because large, systematic replication projects measured it: 36% of 100 studies in OSC 2015, 50 to 54% of 28 findings in Many Labs 2, and about half of 274 findings in SCORE (2026). Each project repeated published methods with bigger samples and, in later projects, preregistered plans.
Which fields of psychology are most affected?
Social psychology fared worst in OSC 2015 (about 25% replicated) and cognitive psychology better (about 50%). Effects that depend on subtle cues, such as social priming, failed most often; basic cognitive effects measured within the same person held up more often.
Does power posing work at all?
The hormonal and risk-taking claims did not replicate: Ranehill and colleagues found no effect on testosterone, cortisol or risk in 200 people. Participants did report feeling more powerful, so a small effect on self-reported feelings remains; whether that changes performance is not established.
Does a failed replication mean the original was fraud?
Almost never. Most failures come from small samples, flexible analysis and publication bias. SCORE's authors stress that a single failed replication does not prove the original wrong; it lowers confidence in it.
Sources
- Open Science Collaboration: Estimating the reproducibility of psychological science, Science 349 (2015)
- Klein et al.: Many Labs 2: Investigating Variation in Replicability Across Samples and Settings, AMPPS (2018)
- Klein et al.: Investigating Variation in Replicability: A "Many Labs" Replication Project, Social Psychology (2014)
- Hagger et al.: A Multilab Preregistered Replication of the Ego-Depletion Effect, Perspectives on Psychological Science (2016)
- Vohs et al.: A Multisite Preregistered Paradigmatic Test of the Ego-Depletion Effect, Psychological Science (2021)
- Wagenmakers et al.: Registered Replication Report: Strack, Martin, & Stepper (1988), Perspectives on Psychological Science (2016)
- Coles et al.: A multi-lab test of the facial feedback hypothesis by the Many Smiles Collaboration, Nature Human Behaviour (2022)
- Dana Carney, UC Berkeley Haas: My position on "Power Poses" (2016; source for the story's sample size, exclusions and p-values)
- Ranehill et al.: Assessing the robustness of power posing, Psychological Science (2015)
- Watts, Duncan & Quan: Revisiting the Marshmallow Test, Psychological Science (2018)
- Aarhus University: Replicability under the microscope (SCORE in Nature) (2026)
- Constructor University: Landmark study finds half of published social science replicates (2026)
- Wikipedia: Replication crisis (OSC 2015 breakdown by field)
