ReproHack: Practicing Research Reproducibility

.title[
# ReproHack: <br/> <i>Practicing Research Reproducibility</i>
]
.author[
### Alessandro Gasparini · alessandro.gasparini(at)ki.se
]
.date[
### Open Access Week · 2022-10-24
]

---

# Introduction

We have seen in the previous talk examples of research reproducibility in practice.

Specifically:

- Evidence of a higher share of reported vs expected positive findings;

- _Researcher degrees of freedom_;

- _Multi-analyst_ studies.

---

# The Replication Crisis

.small[
PubMed Query: '_"replication crisis" OR "replicability crisis" OR "reproducibility crisis"_'. 
<br/>
Data extracted on 2022-10-18.
]

---
class: middle, center, inverse

# What can I do as a researcher?

---
class: middle, center

# Learn

---
class: middle, center

# Advocate

---
class: middle, center

# Practice

---
class: middle, center, inverse

## The previous talk mentioned <br/> _practice_ and _community_.

## <br/> This is what I will be talking about here.

---

# The Importance of Practice

According to Swedish psychologist [K. Anders Ericsson](https://en.wikipedia.org/wiki/K._Anders_Ericsson):

> [...] we argue that the differences between expert performers and normal adults reflect a life-long period of deliberate effort to improve performance in a specific domain.

.footnote[
	K. Anders Ericsson _et al_.
	_The Role of Deliberate Practice in the Acquisition of Expert Performance._ 
	Psychological Review 1993, Vol. 100. No. 3, 363-406
]

Three key points:

* [...] **expert performers**

* [...] **life-long period**

* [...] **deliberate effort**

---
class: middle, center

---
class: center, middle, inverse

# Enter _ReproHack_

---
class: center, middle

# What is a ReproHack?

---

# ReproHack: A Sandbox Environment for Practicing Research Reproducibility

---
class: middle, center

---
class: middle

> During a ReproHack, participants attempt to **reproduce** published research of their choice from a list of proposed papers with **publicly available** associated **code** and **data**.

---

> .grey[During a ReproHack, participants attempt to **reproduce** published research of their choice from a list of proposed papers with **publicly available** associated **code** and **data**.]

> Participants get to work with other people’s material in a low-pressure environment and **record their experiences** on a number of key aspects, including **reproducibility**, **transparency** and **reusability** of materials.

---

> .grey[During a ReproHack, participants attempt to **reproduce** published research of their choice from a list of proposed papers with **publicly available** associated **code** and **data**.]

> .grey[Participants get to work with other people’s material in a low-pressure environment and **record their experiences** on a number of key aspects, including **reproducibility**, **transparency** and **reusability** of materials.]

> At the end of the day we **regroup**, **share** our experiences and give **feedback** to the authors.

---
class: middle, center

# Note: _ReproHacks are by no means an attempt to criticise or discredit work._

---
class: middle, center

---

---

# Benefits of ReproHack

### For authors:

* Feedback on the reproducibility of your work

* Appreciation for your efforts in making your work reproducible.

* Opportunity to engage others with your research.

### For participants:

* Practical experience in reproducibility with real published materials.

* Opportunity to explore different tools and strategies.

* Opportunity for meaningful contributions.

* Inspiration to work more openly.

---

# ReproHack: <br/> Building Communities of Practice in Research Reproducibility

* Help build **capacity** in reproducibility throughout the research community.

* Highlight community **value** of reproducibility beyond validation of results.

* Help the community **evaluate** how successful current practices are and for what purpose.

* Help **identify** what works and where the most pressing weaknesses in our approaches are.

---
class: center, middle

---

# I'm Sold! 
# How Do I Get Involved?

* Check out the website: https://www.reprohack.org

- Resources,

- Current events,

- ...

* Join the Slack: https://reprohack-autoinvite.herokuapp.com

* Join the discourse on Twitter: https://twitter.com/reprohack

* [Organise](https://www.reprohack.org/organiser_guidelines) your own ReproHack!

---
class: center, middle

---

.footnote[
Slides available online: https://tinyurl.com/oaweek-ki-2022
<br/>
Illustrations created by Scriberia with The Turing Way community.
<br/>
Used under a CC-BY 4.0 licence. 
DOI: [10.5281/zenodo.3332807](https://doi.org/10.5281/zenodo.3332807)
]