What if we told you that many published research studies are false? In recent years the scientific community has been shaken by the realization that ‘not all that gets published is gold’. This cold shower hit academia in various forms but probably the most striking example has been the so-called replication crisis in psychology: the mass inability to reproduce established experimental effects through repeated experimentation.

In this workshop we discuss and demonstrate how choices made in experimental design and analysis may result in statistical issues and/or failure to replicate. We also introduce several easy-to-use open source tools that can better detect statistical inconsistencies in research you consume (or produce). Taken together, delegates can expect to leave this workshop with a better understanding of current research issues, as well as an improved ability to plan experiments and evaluate (reported) statistical analyses.

As psychologists we present our content through the lens of the behavioural sciences. Delegates concerned with work involving analysis of human data may therefore find this workshop particularly interesting. Topics and tools include:

• P-values and effect sizes
• Statistical power analysis
• Questionable Research Practices
• GRIM test to evaluate reported means
• SPRITE test to evaluate distributions of reported data

All simulations and tools that we present are implemented in Python. Please bring a laptop if you wish to follow our practical demonstrations. We also invite you to bring along any papers or results that you may wish to scrutinise using the presented techniques!

What do participants need to bring?

Please bring laptop with Python3 installed.

Workshop Organisers: Ozan Vardal, Valerio Bonometti and Myat Aung