Workshop – Replication of Crises: Psychology in Times of Epistemic Upheaval
Luebeck Colloquium on Psychological Humanities
Psychology might be broken, some skeptics warn (Woolston, 2015). It might even be trapped in a horrible space somewhere between the third and fourth circle of hell (Neuroskeptic, 2012). To say the least, psychology is enduring an age of epistemic upheaval. The name of this hell is “replication crisis”. Having already affected a range of disciplines in the medical and the life sciences, the replication crisis reached psychology in 2011, with a paper on false positive psychology (Simmons, Nelson, & Simonsohn, 2011). Since this time, the crisis has grown (Dominus, 2017). In a replication of 100 experimental and correlational studies published in psychology journals, only thirty-six percent of the results proved replicable (Open Science Collaboration, 2015). Many other major psychological findings could not be replicated, especially in social and clinical psychology: from the broadly published effects of ‘power posing’ to some well-known finding on behavioral priming and the famous Marshmallow test.
The reactions within psychology have been vehement but not surprising. Many scientists are calling for more methodological rigor, for stricter statistical approaches that prevent p-hacking, or for the use of a replication-index.
In this workshop, we do not aim to identify new methodological tools to ‘repair’ psychology. Instead, we will place the replication crisis in its scientific, historical and social context. Therefore, we invite psychologists and historians as well as philosophers and sociologists of science to reflect on the epistemic dynamics within the discipline. The workshop will also discuss the effects of this crisis on public debates, on scientific discourses, and on practices of psychological research.
To this end, we welcome theoretical papers as well as historical analyses or specific case studies of (non-)replicable psychological experiments.
One major question, which the workshop seeks to address, regards the continuities and/or discontinuities of the replication crisis: On the one hand, there are considerable continuities. The current crisis can be understood as just another example of the eternal recurrence of methodological crises – or at least of crisis discussions – in psychology (Stam, 2018). Moreover, focusing on replicability is not a new phenomenon in psychology, but continues older debates on epistemic ideals that go back to the 19thcentury (Mülberger, 2018). In view of these continuities, we could indeed speak of a replication of crises in psychology. On the other hand, however, the concerns regarding replication might indicate an epistemological rupture reflecting changed practices in, and ideas of, psychology. There are indications of at least two kinds of discontinuities. Firstly, the crisis might mark the endpoint of an epistemic shift from attending to experimental systems in the laboratory to the post-experimental engagement with laboratory data. Secondly, the crisis might imply a reversal of criteria of scientific prestige: While failed replications and non-significant results were little appreciated before, they contain a considerable potential to gain scientific attention now. In polemic terms, a researcher who fails to replicate a major experiment is more likely to attract recognition than one, who publishes new psychological findings.
Here is an open list of questions we would like to discuss in the workshop:
- What is the genealogy of the current crisis in the long history of similar problems in psychology?
- When and why did the idea of replication emerge as an epistemic ideal, under which circumstances did its relevance fluctuate and how did it gain importance in psychology?
- How do concerns about replication and replicability reflect broader scientific assumptions and ideals of robustness and stability?
- How does the issue of replication in psychology relate to the situation in other disciplines like medicine, the life sciences, environmental studies, or climate science?
- How do we as critical scholars deal with the “replication crisis” and with what intentions do we evaluate the focus on this issue in the debate on psychology?
- Is replication “overrated” (Feest, 2018), because it is based on a problematic Popperian paradigm of experimental psychology?
- Can the replication of data alternatively be understood by theoretical approaches from ‘new materialism’ as a phenomenon of posthumanist performativity (Barad, 2003)?
- What are the political and cultural contexts of the replication crisis and how does it relate to the debate on ‘alternative facts’ or the rhetoric of a ‘war on science’?
- How do gender issues play out in the notion of a replication crisis and ideas of “soft” versus “hard” science?
- Which larger cultural concepts, metaphors, and ideas resonate with this “crisis” discourse?
Barad, K. (2003). Posthumanist Performativity. Toward an Understanding of How Matter Comes to Matter. Signs, 28(3), 801-831.
Dominus, S. (2017, Oct. 18th 2017). When the Revolution Came for Amy Cuddy. New York Times Magazine.
Feest, U. (2018). Why Replication is Overrated. Paper presented at PSA2018: The 26th Biennial Meeting of the Philosophy of Science Association, Seattle.
Mülberger, A. (2018). When and Why did Psychologists start to Worry about Replication? Paper presented at the European Society for the History of the Human Sciences, Groningen, Netherlands.
Neuroskeptic. (2012). The Nine Circles of Scientific Hell. Perspectives of Psychological Sciences, 7(6). doi:10.1177/1745691612459519
Open Science Collaboration. (2015). Estimating the Reproducibility of Psychological Science. Science, 349(6251), 943. doi:10.1126/science.aac4716
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359-1366.
Stam, H. J. (2018). Once More with Feeling: The Eternal Recurrence of the Reproducibility Crisis in Psychology. Paper presented at the European Society for the History of the Human Sciences, Groningen, Netherlands.
Sturm, T., & Mülberger, A. (2012). Crisis Discussions in Psychology—New Historical and Philosophical Perspectives. Studies in History and Philosophy of Biological and Biomedical Sciences, 43, 425-433.
Woolston, C. (2015). Online Debate Erupts to Ask: Is Science Broken? Nature, 519(7544). doi:10.1038/519393f
Organizers: Eva Barlösius, Cornelius Borck, Uljana Feest, Lisa Malich, Jonas Obleser, Torsten Wilholt
with organizational support of Anna Weilandt and Kathrin Langkau.
Image/ Statistical Graph: Patrick Bartlein