Critical Metascience Roundup
Update #5
Summary
Average replicability rates
Theories don’t die!
Is the replication crisis overblown?
Postdoctoral Fellowships
Partial replications
Average Replicability Rates
Berna Devezer and Erkan Buzbas published a preprint disputing the idea that a high average replication rate indicates a healthy literature and a low rate indicates a crisis. They argue that:
Aggregating replicability rates across heterogeneous literatures produces averages that conflate incommensurable experimental regimes and lack a stable scientific interpretation.
The common metascientific practice of referring to an average replicability rate for a diverse population of results (e.g., Camerer et al., 2016; Collaboration, 2015), sometimes even across disciplines (e.g., Tyner et al., 2026), is statistically fraught and misleading.
Devezer, B., & Buzbas, E. O. (2026). The difference between “replicable” and “not replicable” is not itself scientifically replicable. ArXiv. https://doi.org/10.48550/arXiv.2604.26268
Theories Don’t Die!
Adam Mastroianni wrote a Substack post on the futility of trying to kill (falsify) scientific theories. He argues that, despite our best efforts, theories about power posing, ego depletion, growth mindset, stereotype threat, etc. are hard to falsify and tend to linger on in some form or another.
It might seem like the way to resolve these disputes is to weigh up all the evidence, meta-analyze all the data, deploy your p-curves, your moderator analyses, and your tests for heterogeneity, maybe even run a big, multi-lab, preregistered replication. Do all that and then we’ll finally know whether these effects are real or not!
This is a trap. We have spent the past decade doing exactly those things, and yet here we are. Clearly, no amount of data-collecting, number-crunching, or bias-correcting is going to lay these theories to rest, nor will it return them to the land of the living.
Is the Replication Crisis Overblown?
Bruce Lanphear wrote a Substack post on the dangers of making sweeping claims about a universal replication crisis:
He concludes that:
If we want to preserve confidence in science, we must speak precisely. Not all findings are equally unstable. Not all literatures are equally fragile.
Postdoctoral Fellowships
Bart Penders posted on BlueSky that “we are joining forces with our metaresearch centre and are offering ourselves as hosts for potential MSCA [Marie Skłodowska-Curie Actions] postdoctoral fellows.”
We are especially keen on candidates and projects that span qualitative and quantitative #methods, pioneer and reflect on new dimensions of #openness and #openscience and target various conceptualisations of improvement and reform in science.
For more information about MSCA postdoctoral fellowships: https://marie-sklodowska-curie-actions.ec.europa.eu/actions/postdoctoral-fellowships
For enquiries, please contact Bart at b.penders@maastrichtuniversity.nl
Partial Replications
Sven Ulpts will present his interview research on “partial replications” at the European Association for the Study of Science and Technology’s 2026 conference.
During the interview reflections researchers state they employ “partial replications” in which they take parts of a previous study and incorporate it into their own research. They state that direct or exact replication is not worth the effort while “partial replication” allows them to partially build on previous research, while still exploring something novel and following their own interests.
Abstract of Sven’s presentation: https://nomadit.co.uk/conference/easst2026/paper/102612
Part of the Open Panel “Making Order in Science through Reform: The Politics of Replication and Research Information Infrastructures”: https://nomadit.co.uk/conference/easst2026/p/20333
More info about the EASST2026 conference: https://easst.net/conferences/easst2026/
List of Critical Metascience Articles
For an up-to-date list of research articles in this area, please visit: https://sites.google.com/site/markrubinsocialpsychresearch/replication-crisis/list-of-critical-metascience-articles