Beware the Threshold: Looking beyond averages

by Loic Menzies, 09 March 2022
Representation of a graph with people

I don’t think Brandon expected to get a GCSE certificate. Let alone in Year 10. I’d entered him early because I had a feeling he wouldn’t make it through to the end of his GCSEs, but when he opened the envelope from the exam board he was confused, not knowing whether to be elated or disappointed. Brandon had got an E.

“That’s a fail isn’t it sir?” he asked me. 

Well no, not really. Although I’m a fully paid- up member of the ‘high expectations squad’, all things considered, I think he’d done pretty well not to end up with a U. 

The fine grain of who gets a D, E or F in old money (a 1, 2 or 3 under the new grading system), doesn’t get a lot of attention - despite these technically being pass grades and level 1 qualifications. In fact, a 2019 report from the Office of the then Children’s Commissioner described pupils with these grades as leaving school with ‘nothing’. Yet distinctions between these grades represent real differences in attainment and offer insight into some of our most vulnerable learners’ progress - or lack thereof. 

If we want an education system that secures equity, then pupils on the edge of the attainment distribution can’t remain ‘forgotten’ and this means policy makers need to fixate less on averages and thresholds. Only then can they understand what’s going on for pupils across the whole distribution. The shift to ‘Progress 8’ as the headline measure of school performance in England (rather than percentages reaching thresholds), represented some recognition of this, but problems remain, with many audiences focusing on whether a school has positive or negative score, often ignoring the confidence intervals.

The problem is markedly worse at the ‘left-hand side’ of the distribution; no end of media coverage and politicians speeches count the top achievers’ haul of A*s, or praise the schools sending the most students’ to Oxbridge. Is it therefore any surprise that policy makers favour high-profile announcements that might boost performance for a few pupils at the top - even if they risk hampering equity? Some media outlets have made an effort to publish alternative league tables, highlighting pupils who achieve less traditional measures of success against the odds, but these are rare.  

It’s not fair to make pupils like Brandon feel like they’ve failed. Moreover, from a policy maker’s perspective, if information about young people at the margins of the distribution is ignored, then decision-making is impaired. 

This can be seen most clearly with young people who have profound Special Educational Needs and Disabilities (SEND) - some of whom have conditions that mean they are unlikely to achieve success on traditional measures. The invisibility - to policy makers, of these young people’s achievements mean that stand-out successes as well as glaring failures go unnoticed. It then becomes difficult to identify where the system is working well and spread good practice, or to tackle underperformance where necessary. Instead, the focus becomes ‘processing of cases’ and organising meetings rather than meeting needs and supporting progress. Yet as one SEND advocate puts it, “meetings are not outcomes”. 

One of the big themes in my book “Young People on the Margins: Priorities for Action in Education and Youth” is that there will always be young people at the margins - but if we don’t leave them out of the picture then this needn’t mean they are marginalised.

Map-making often starts with triangulation from a high vantage point, but if the surveyor only looks at one part of the landscape, then their partial view will be of limited use. Policy makers who obsess about averages, or who only look at a threshold, are making exactly this mistake. In order to fulfil their responsibility towards equity in education, then it's time they checked their peripheral vision and paid attention to the full panorama.

Over the coming months, Loic Menzies and Cambridge Assessment Network will be convening a series of expert round tables and discussion seminars on the topic of equity in education. Our first event in February asked, 'What information do policy makers need to help them map the way to educational equity?’ 

This is the first in a four-part blog series setting out the key considerations to come from the event. 

The aim for the series is to build an incredibly rich body of thinking, which participants will have played a critical role in helping to develop.

You can catch up with the latest event over on our YouTube channel and make sure you're signed up to our mailing lists to be find out how you can get involved in the next one

Before you go... Did you find this article on Twitter, LinkedIn or Facebook? Remember to go back and share it with your friends and colleagues!

Related blogs

Key bloggers

Tim Oates Blogger
Tim Oates

Director of Assessment and Research

Research Matters

Research Matters 32 promo image

Research Matters is our free biannual publication which allows us to share our assessment research, in a range of fields, with the wider assessment community.