16 April 2021
New research from Cambridge Assessment concludes that the extent of generosity shown by schools and colleges in determining GCSE and A level grades in England in summer 2020 was logical, rational and justifiable, but is cautious about the implications for grading standards in future.
The paper by Tom Benton, Principal Research Officer, is published in the latest edition of Research Matters, Cambridge Assessment's free biannual research publication. It says that schools and colleges recognised their ability to determine grades was less reliable than if exams had taken place last summer and wanted to avoid students being awarded a worse grade than might otherwise have been the case. He describes the generosity they exhibited as rational in the circumstances. However, he notes that it is impossible to know which individual students benefitted from or were disadvantaged by the switch from exams.
Benton says:
“Once we know that the reliability of an assessment is lower than normal, the natural human inclination is to try to ensure that students do not lose out. Reduced reliability will increase the chances of a student ending up with a better grade than they would have achieved with a more reliable assessment.”
The paper’s findings are based on an analysis of the extent to which grade boundaries might have had to change in 2019 to deliver different choices about standards. The analysis uses the difference between forecast A level grades submitted to the exam board OCR by teachers in 2014 and actual exam grades that year. This direct comparison for the same students in the same subjects is the best source of information for considering the reliability of centre assessment grades in 2020. In one set of circumstances based on these previous estimates of the reliability of teacher grading – where grade boundaries are set to ensure that the overall proportion of students awarded a lower grade than merited by their true ability does not increase as the reliability of assessment falls – the proportion of students awarded A and above at A level could have been expected to increase last year from 25.5% to 36.2%. As it turned out, schools and colleges submitted grades to exam boards that increased the proportion awarded A and above from 25.5% overall in 2019 to 37.6% in 2020.
Benton adds:
“The final distribution of grades in summer 2020 was similar to what might be expected from a logical application of giving students the benefit of the doubt from a position of uncertainty about how they would have performed in real exams. This may indicate that many teachers had a natural intuition for how confident they could be in their own estimates and applied a logical level of benefit of the doubt to help ensure that students were not disadvantaged relative to a normal year. Of course, the similarity in results may be purely coincidental. However, it does illustrate how major changes in grade distributions between years need not necessarily in themselves indicate inappropriate decisions. If our aim is to protect students from any adverse effects of added unreliability, a change in grade distributions is a logically justifiable result.”
The paper concludes by considering arguments that the generous distribution of GCSE and A level grades in England from 2020 should be carried forward into the future when ‘normal’ examinations resume, as not doing so would, according to some, be unfair to subsequent cohorts of students. However, Benton argues that this fails to recognise the possible role of benefit of the doubt in teachers assigning grades in 2020, which helped protect individual students from being under-rewarded.
Responding to the research, Jill Duffy, CEO of OCR, a leading UK awarding body and part of the Cambridge Assessment Group, said:
“The means of determining grades in 2021 is different from normal and from 2020, so the overall grade distribution in 2021 is likely to look different from 2020 and previous years. Teachers should ensure that they take an evidence-based approach to determining grades this summer in line with the detailed guidance and training provided by exam boards and try to be as accurate as possible. Decisions about potential must not factor into students’ grades; if a student is currently performing consistently at a grade B standard, they should be awarded a grade B. This evidence-based approach, together with both the internal quality assurance schools and colleges will undertake and the external quality assurance that exam boards working together will conduct, will provide confidence in the grades awarded this Summer.”