Open Research and Reproducibility

Introduction

Welcome to the Open Research and Reproducibility workshop. Like the other workshops in this series, this one involves a mix of recorded videos, narrative, and links to various resources for you to explore and to read.

Open Research (A Brief History)

First I’d like you to watch the following video where I describe the history of the replication crisis and the issues which have motivated a move towards the adoption of open and reproducible research practices. I delivered a version of this talk as my keynote at the Collaborations Workshop 2020 event run by the Software Sustainability Institute. I cover the so-called replication crisis in the biomedical sciences, issues around open research, and summarise some of the initiatives (including the UK Reproducibility Network) that have been established to address the fundamental problems around open research, transparency, and reproducibility in science.

  

  

Slides

  

Before watching the next video, please have a read through the following paper by Ionnidis (2005) which arguably started the conversation around reproducibility that has had such an impact on research in Psychology and across the Biomedical sciences for the last few years. Clicking on the image below will take you to the paper.

Ionnidis (2005)

  

  

Bishop (2019)

This post by Dorothy Bishop in 2019 nicely captures the situation a number of years later. Clicking on the image below will take you to the paper.

  

How to do Reproducible Research

One of the biggest challenges facing researchers who are used to the old way of conducting research is that they feel that they don’t have the knowledge or technical skills to adopt open and reproducible research practices. But it’s not that hard! Before you run your experiment, you can pre-register your hypotheses so that when you come to analyse and write-up your results, you can demonstrate that your predictions really were made in advance of data collection. You can also make your research data open (and FAIR) alongside your code so that others can recreate your analyses. And by posting your research article on a pre-print server (such as PsyArXiv or bioRxiv) before submission to a journal, you make your research findings available to all. The adoption of open source software such as R, also means that any research findings you produce can be re-produced by others who can access your data and code. This principle of using open tools to allow us to produce open (and reusable) data and code is the fundamental philosophy behind all of the workshops in this unit. Have a look at the video below where I talk about how we can adopt open and reproducible research practices.

  

  

Slides

  

Crüwell et al. (2019)

Here’s a great guide to the various things you can do to make your own research more open. Just click on the image to open the paper.

  

Haeffel (2022)

This thought provoking paper by Gerald Haeffel suggests that psychology needs to focus more on theory development (and encourage the publication of results that refute theories).

Improve this Workshop

If you spot any issues/errors in this workshop, you can raise an issue or create a pull request for this repo.