The philosophy of the GCSE is sometimes said to be that of 'rewarding positive achievement', but what does this mean, exactly? Here, the Director of our Research Division, Tom Bramley, unpacks the UK's exam grade boundary setting system. This article first appeared in the TES and you can read the original here.
Some members of the Maths community are reported to have concerns about the low grade boundaries on the UK higher tier GCSE maths exams. One
quote from this article in the TES was: "Any system where you can get 87% on a paper wrong and still get a GCSE has to be flawed"; the reference being to a higher tier paper where the boundary for a level 4 (equivalent to a grade C) was at 13% of the maximum.
The philosophy of the GCSE is sometimes said to be that of ‘rewarding positive achievement’, but what does this mean, exactly? It could mean that after 11 years of compulsory education all students have learnt something in the subjects they’ve studied, and it is important to recognise this by having a grade scale that covers all levels of achievement such that virtually no-one is regarded as having failed.
But if the exam needs to cater for all levels of achievement then inevitably those at the lower end will not be able to achieve much on it, and their grade 1 (equivalent to a grade G) will as a consequence signal what they don’t know rather than what they do. If a grade 1 required 50% of the marks to be gained, for example, then the remaining eight grade boundaries would be bunched closely together on the mark scale between 50% and 100%. Small differences in achievement (in terms of marks) could lead to large differences in grade outcomes, and the exam would be very easy for the most able students.
The solution to this problem is to have ‘differentiated assessment’ – exams of different difficulty targeted at students of different ranges of achievement level. With fewer grades to cover, the lowest grade boundary can be set at a higher proportion of the maximum mark, and the exam can allow those at the lower levels to show what they can rather than can’t do. But differentiated assessment can be difficult politically because it tends to provoke cliches in public debate along the lines of the dangers of a ‘two-tier system’ creating ‘second-class citizens’ who could end up being ‘consigned to the scrap heap of history’. This is allegedly what happened with the O level and the CSE.
The genius of the GCSE was to allow all achievement to be reported along a single grade scale while (in some subjects) including differentiated assessment in the form of tiered papers. Since 2006 all tiered subjects have had a Foundation tier and a Higher tier – a two-tier system if ever there was one – so why was this so readily accepted? The answer is that the two tiers have a range of overlapping grades and the politically important grade 5 is obtainable on both of them. If the Foundation tier covered grades 1-4 and the Higher tier grades 5-9 then this would surely be seen as a ‘two-tier system’ in the negative sense.
However, it might be argued that avoiding public disquiet is the only good thing about overlapping grades. They immediately raise a difficult problem for the exam boards (and regulator) who have to ensure that the overlapping grades in some sense mean the same thing regardless of which tier they were obtained on. (The tier taken is not reported on the GCSE certificate). The exam boards achieve statistical alignment of the overlapping grades via scores on questions that are common to both tiers. However, the overlapping grades cannot mean the same thing in terms of inferences about knowledge and skills if the Higher tier course contains material that is not taught on the Foundation tier course - which it does in both Maths and Science GCSE. A grade 5 on the Foundation tier therefore means relatively good achievement on a subset of the material covered on the Higher tier course, while a grade 5 on the Higher tier means relatively less achievement on all the material. If you are only aware of a student’s grade and not the tier, you cannot make sound inferences about what they know and can do. Furthermore, in a system allowing overlapping grades, if higher tier papers are intended to be challenging for students working at the level of the higher grades, they necessarily will contain more difficult questions than an untiered exam would. It will consequently be more difficult for those working at the level of the overlapping grades to gain marks and hence the grade boundaries for those overlapping grades will be at a lower percentage of the maximum than on an untiered exam.
Over the years a variety of tiering models have been proposed and used in practice. One model, the ‘adjacent levels model’, was used for many years in Scotland. Three levels of assessment were available with non-overlapping grades. Students would enter for two adjacent levels and would be awarded their best result from the two. I was impressed with the simplicity of this approach and recently published
a paper considering the advantages and disadvantages of using a version of it in GCSE Maths.
Other research on tiering at Cambridge Assessment has looked at:
the effect of tiering on student aspirations; whether tiering
‘caps’ the achievement of students entered for the Foundation tier;
teachers’ views about tiering; and how the
form of assessment (linear vs modular) and student characteristics influence tiering decisions. This research has shown that, on the whole, teachers make good entry decisions on behalf of their students, and there is little evidence that students’ achievement was capped by tiering in the old ‘unitised’ or ‘modular’ pre-reform GCSEs.
Even in untiered exams, or tiered exams with non-overlapping grades, there is still some uncertainty in what any individual student with a particular grade knows or can do, since knowledge of the grade alone does not tell you which questions were answered correctly or incorrectly. In the end, therefore, we have to accept that there are trade-offs to be made. The current system with overlapping grades in those few subjects that are tiered seems to be broadly favoured by students, teachers, parents and politicians. The price we pay is the uncertainty in the meaning of the overlapping grades in terms of inferences about what students know and can do, and the risk that on the higher tier papers, which are supposed to be challenging for the most able students, the lowest grade boundary will be at a very low proportion of the maximum mark.
Tom Bramley
Director, Research Division, Cambridge Assessment
Before you go... Did you find this article on Twitter, LinkedIn or Facebook? Remember to go back and share it with your friends and colleagues!