Research on Research, Open Infrastructures, Adaptive Preregistration, and Science Reform
The Journal of Research on Research
The Journal of Research on Research has recently been launched, with Gemma Derrick as the Editor-in-Chief, and Serge Horbach, Bart Penders, and Tony Ross-Hellauer as Editors.
Why call it “research on research”? In their Editorial, Derrick et al. explain that:
We have purposefully chosen a term which lays aside questions of epistemological inclusion or exclusion of contributions and instead embraces all ways of doing, knowing and understanding how research is practiced, produced and governed. The term, as opposed to adopting the more politically popular ‘meta-science’, includes perspectives from fields with long-standing traditions in the study of research: scientometrics; science and technology studies; computer science; library sciences; higher education; evaluation studies; the philosophy, sociology, anthropology; history of science, as well as, and more importantly, entire communities of metascience and meta-research.
Derrick, G., Horbach, S. P. J. M., Penders, B., & Ross-Hellauer, T. (2026). Introducing the Journal of Research on Research (J·ROR): aligning constellations and communities. Journal of Research on Research, 1(1). https://doi.org/10.1080/30700485.2026.2680358
The Politics of Open Infrastructures
Katja Mayer, Astrid Mager, and Renée Ridgway have edited a new open access book titled “The Politics of Open Infrastructures: Power, Governance, and Justice in Digital Knowledge Practices.” Substack authors include Celina Strzelecka, Stefano Crabu and Leonhard Dobusch.
Bringing together scholars from science and technology studies, critical data studies, media studies, organisation studies, arts-based research and political sociology, this edited volume explores openness as an ongoing socio-technical process rather than a fixed ideal.
The book is associated with a companion blog here.
Over the coming months, we will invite our authors to reflect on their chapters, share additional materials and discuss the questions that emerged during the making of the book. We also want to bring in other scholars, practitioners and experts working on open knowledge infrastructures, digital commons, open source, public data systems, citizen science, cultural infrastructures and related fields.
Meyer, K., Mager, A., & Ridgway, R. (2026). The politics of open infrastructures: Power, governance, and justice in digital knowledge practices. https://doi.org/10.11647/OBP.0528
Adaptive Preregistration
Jeremy Fox has an excellent blog post that provides a critical analysis of “adaptive preregistration” (Gould et al., 2026):
Gould et al. … say that, once you make a decision that you anticipated needing to make, or after you deviate from your preregistered plan in some way, you can file an updated preregistration. This is the ‘adaptive’ part of ‘adaptive preregistration.’ But I confess I don’t see what’s accomplished by labeling all the deviations from your preregistered plan as ‘adaptations’ and then repeatedly updating your preregistration. How is that any different from having a single preregistered plan and then describing all the deviations from the plan in text boxes in the resulting paper? Or indeed, how is it any different from not having a preregistered plan at all?
Fox also considers the evidence for preregistration more generally:
When it comes to preregistration specifically, it feels to me like the time is right for its advocates–in psychology, in ecology, and in other fields–to take a step back and pause to reflect. We now have a lot of evidence from psychology as to whether preregistration will actually take off, in a way that actually improves scientific practice rather than merely being an empty box-ticking exercise (or worse, a way to mislead readers). To my eyes at least that evidence looks quite negative.
Fox, F. (2026, June 8). Defining preregistration down. Dynamic Ecology. https://dynamicecology.wordpress.com/2026/06/08/defining-preregistration-down/
Science Reformers
Sheena Bartscherer will present her work on the reflexivity and reflectivity of science reformers at 2.00pm on Wednesday 17th June as part of a Metascience Research Seminar at the University of Sussex Business School. Register for free online attendance here.
I will present my newly started research consortium “Reforming Science”. The project is looking to investigate the different actors who are currently, but also historically advocating for science reforms, how they discuss these matters publicly, how critiques are interacted with (reflexivity / reflectivity), and how they organise themselves (e.g., as a social movement / research speciality such as Metascience).
Bartscherer, S. F. (2026, June). Discussing science reform: Investigating the reflexivity & reflectivity of actors advocating for/against reforming science. https://www.eventbrite.co.uk/e/metascience-research-seminar-dr-sheena-f-bartscherer-tickets-1989914011375
Questioning Some Metascience Assumptions
I’m grateful to Jan Feld for the invitation to present some of my work to the Association for Interdisciplinary Metaresearch & Open Science (AIMOS). My presentation is titled “Questioning Some Metascience Assumptions,” and it will take place online on Thursday 13th August at 7.00am British Standard Time (but also be recorded!). You can register for free here.
Metascience uses a scientific approach to understand and improve the scientific approach! In this presentation, I question some common assumptions in modern metascience. For example, I question whether exploratory research is more “tentative” than confirmatory research and whether questionable research practices such as HARKing (hypothesizing after the results are known) and undisclosed multiple testing are problematic. I discuss these and other metascience assumptions and argue that they may not be as problematic as commonly assumed.
Rubin, M. (2026, August). Questioning some metascience assumptions. Association for Interdisciplinary Meta-Research & Open Science (AIMOS), Victoria University of Wellington, New Zealand. https://events.humanitix.com/questioning-some-metascience-assumptions-with-mark-rubin
“10 Years Ago…”
In their highly cited article (470+ citations), Chris Crandall and Jeff Sherman considered the distinction between direct (“exact”) and conceptual replications.
In direct replications, the unit of analysis is the previously observed effect. The question to be addressed is to what extent can that effect be repeated? Such a replication cannot tell us any more about the meaning of the effect or any more about why the effect is obtained.
By contrast, ideas are the unit of analysis in conceptual replication. The question becomes not whether a specific finding may hold, but whether a theory can be retained in the face of multiple and variable tests of its hypotheses.
The authors concluded with a call for a pluralist approach that supports both direct and conceptual replications:
Some scientists want—and need—exact replications to generate confidence in their experiments. These scientists should conduct them, they should recommend publication of them in the review process, and they should read, cite, and enjoy them. But these same scientists should not require the same behaviors of others, make a narrow type of scientific practice necessary for publication, nor expect all others to share their values. We recommend the same tolerance for those who prefer conceptual replications. A lack of dissent and diversity in a scientific community is a sure prescription for lethargic progress.
Crandall, C. S., & Sherman, J. W. (2016). On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology, 66, 93-99. https://doi.org/10.1016/j.jesp.2015.10.002