How much can we trust the reporting of key statistics? Not just the headline findings, but the nitty-gritty detail underpinning them? Those things that it’s important for consumers of data to know if they want to form their own independent judgement about the strength of the evidence available.

In this talk I will discuss what I consider to be a worrying lack of transparency surrounding some aspects of the reporting of the influential PISA study for the UK.

I will highlight how there are many anomalies with the data, with the combination of non-response, exclusions from the test and technical details surrounding eligibility criteria leading to total non-participation rates of around 40% (amongst the highest level anywhere in the world). I will then discuss how this leads to substantial uncertainty surrounding the PISA results, with clear evidence of bias in the sample for certain parts of the UK. I will conclude by discussing how more transparent reporting of the technical details underpinning PISA is needed, at both a national and international level.

About the speaker:

Professor John Jerrim, Professor of Education and Social Statistics, UCL

John Jerrim is a Professor of Educational and Social Statistics at UCL Institute of Education. He has analysed PISA data for over ten years, and has worked at the OECD as a Thomas J. Alexander fellow. John was lead author of the PISA 2015 national reports for England, Wales and Northern Ireland and has published more than 20 academic articles using these data.