Joanna Williamson

Williamson, J., & Vidal Rodeiro, C. L. (2023, November 1–4). Performance in secondary mathematics topics pre- and post-reform. [Paper presentation]. Annual conference of the Association for Educational Assessment – Europe (AEA-Europe), Malta. https://2023.aea-europe.net/ 

Vidal Rodeiro, C. L., & Williamson, J. (2023, November 1–4). Evaluating the impact of curriculum and assessment reform in secondary education on progression to mathematics post-16. [Paper presentation]. Annual conference of the Association for Educational Assessment (AEA-Europe), Malta. https://2023.aea-europe.net/

Vidal Rodeiro, C., & Williamson, J. (2023). The impact of GCSE Maths reform on progression to mathematics post-16 Research Matters: A Cambridge University Press & Assessment publication, 36, 25-45. https://doi.org/10.17863/CAM.101743

In England, GCSE (General Certificate of Secondary Education) qualifications offered to students aged 14–16 were recently reformed. For mathematics specifically, the new GCSE aimed to be more demanding, provide greater challenge for the most able students, and support progression to post-16 mathematics. However, there have been concerns that the new GCSE could deter students from further education in the subject and, to date, there has been little research on its impact on participation in and learning of mathematics post-16.

This research compared progression to and performance in post-16 mathematics pre- and post-GCSE reform and found that, contrary to fears about reduced uptake, progression generally increased following the reform. In particular, the increase was higher among those who achieved top grades in their mathematics GCSE than for students with just a pass. Performance in post-16 mathematics was, in contrast to teacher expectations, lower post-reform.

• Foreword Tim Oates
• Editorial Tom Bramley
• The prevalence and relevance of Natural History assessments in the school curriculum, 1858–2000: a study of the Assessment ArchivesGillian Cooke
• The impact of GCSE maths reform on progression to mathematics post-16Carmen Vidal Rodeiro, Joanna Williamson
• An example of redeveloping checklists to support assessors who check draft exam papers for errorsSylvia Vitello, Victoria Crisp, Jo Ireland
• An analysis of the relationship between Secondary Checkpoint and IGCSE resultsTim Gill
• Synchronous hybrid teaching: how easy is it for schools to implement?Filio Constantinou
• Research NewsLisa Bowett

Williamson, J., & Bramley, T. (2022). What’s in a name? Are surnames derived from trades and occupations associated with lower GCSE scores? Research Matters: A Cambridge University Press & Assessment publication, 34, 76–97.

In England, there are persistent associations between measures of socio-economic advantage and educational outcomes. Research on the history of names, meanwhile, confirms that surnames in England – like many other countries – were highly socially stratified in their origins. These facts prompted us to wonder whether educational outcomes in England might show variation by surname origin, and specifically, whether surnames with an occupational origin might be associated with slightly lower average GCSE scores than surnames of other origins. Even though surnames do not measure an individual’s socio-economic position, our hypothesis was that in aggregate, the educational outcomes of a group defined in this way might still reflect past social history.

In line with the research hypothesis, the results showed that the mean GCSE scores of candidates with occupational surnames were slightly lower than the mean GCSE scores of candidates with other surnames. The difference in attainment was a similar size to the difference expected between candidates half a year apart in age, and much smaller than the “gap” between male and female candidates. The explanation for the identified effect was beyond the scope of the current research, but surname effect mechanisms proposed in the literature include the psychological (e.g., implicit egotism), sociological and socio-genetic.

Benton, T., & Williamson, J. (2022). Which assessment is harder? Some limits of statistical linking. Research Matters: A Cambridge University Press & Assessment publication, 34, 26–41.

Equating methods are designed to adjust between alternate versions of assessments targeting the same content at the same level, with the aim that scores from the different versions can be used interchangeably. The statistical processes used in equating have, however, been extended to statistically “link” assessments that differ, such as assessments of the same qualification type that assess different subjects. Despite careful debate on statistical linking in the literature, it can be tempting to apply equating methods and conclude that they have provided a definitive answer on whether a qualification is harder or easier than others.

This article offers a novel demonstration of some limits of statistical equating by exploring how accurately various equating methods were able to equate between identical assessments. To do this, we made use of pairs of live assessments that are “cover sheet” versions of each other, that is, identical assessments with different assessment codes. The results showed that equating errors with real-world impact (e.g., an increase of 5–10 per cent in the proportion of students achieving a grade A) occurred even where equating conditions were apparently favourable. No single method consistently produced more accurate results than the others.

The results emphasise the importance of considering multiple sources of information to make final grade boundary decisions. More broadly, the results are a reminder that if applied uncritically, equating methods can lead to incorrect conclusions about the relative difficulty of assessments.

• Foreword Tim Oates
• Editorial Tom Bramley
• Learning loss in the Covid-19 pandemic: teachers’ views on the nature and extent of loss Matthew Carroll, Filio Constantinou
• Which assessment is harder? Some limits of statistical linking Tom Benton, Joanna Williamson
• Progress in the first year at school Chris Jellis
• What are "recovery curricula" and what do they include? A literature review Martin Johnson
• What's in a name? Are surnames derived from trades and occupations associated with lower GCSE scores? Joanna Williamson, Tom Bramley
• Research News Lisa Bowett

• Foreword Tim Oates
• Editorial Tom Bramley
• A summary of OCR’s pilots of the use of Comparative Judgement in setting grade boundaries Tom Benton, Tim Gill, Sarah Hughes, Tony Leech
• How do judges in Comparative Judgement exercises make their judgements? Tony Leech, Lucy Chambers
• Judges' views on pairwise Comparative Judgement and Rank Ordering as alternatives to analytical essay marking Emma Walland
• The concurrent validity of Comparative Judgement outcomes compared with marks Tim Gill
• How are standard-maintaining activities based on Comparative Judgement affected by mismarking in the script evidence? Joanna Williamson
• Moderation of non-exam assessments: is Comparative Judgement a practical alternative? Carmen Vidal Rodeiro, Lucy Chambers
• Research News Lisa Bowett

Williamson, J. (2022). How are standard-maintaining activities based on Comparative Judgement affected by mismarking in the script evidence? Research Matters: A Cambridge University Press & Assessment publication, 33, 80–99.

An important application of Comparative Judgement (CJ) methods is to assist in the maintenance of standards from one series to another in high stakes qualifications, by informing decisions about where to place grade boundaries or cut scores. This article explores the extent to which standard-maintaining activities based on Comparative Judgement would be robust to mismarking in the sample of scripts used for the comparison exercise. While extreme marking errors are unlikely, we know that mismarking can occur in live assessments, and quality of marking can vary. This research investigates how this could affect the outcomes of CJ-based methods, and therefore contributes to better understanding of the risks associated with using CJ-based methods for standard maintaining. The article focuses on the ‘simplified pairs’ method (Benton et al., 2020), an example of the ‘universal method’ discussed by Benton (this issue).

Jellis, C., Suto, I., Williamson, J. (2021). How well do we understand wellbeing? Teachers’ experiences in an extraordinary educational era. Research Matters: A Cambridge University Press & Assessment publication, 32, 45-66.

COVID-19 has had a huge effect on education. While there has been much interest in the effects of school closures on children, less attention has focused on teachers’ wellbeing. This article describes a small-scale study in which we explored teachers’ experiences and concerns during and after England’s second national school closure in early 2021. Our aim was to improve understanding of how teachers had been impacted in these unprecedented times. 54 teachers in England completed an online survey which was based on a well-established scale of teacher wellbeing. The survey also asked about what would improve teachers’ wellbeing.

Levels of organisational and student interaction wellbeing were reported to be positive both during and after lockdown, but slightly higher after lockdown. By contrast, reported workload wellbeing was slightly negative overall, and slightly lower after lockdown. Strikingly, the issues that most affected teacher wellbeing were not especially connected to lockdown. Teachers were most concerned about the time available to do their jobs and the amount of administration expected of them. Interestingly, some of the longest-serving teachers were amongst those finding that time pressure and administration affected their wellbeing. We conclude that teachers’ longer-term working conditions impacted their wellbeing far more than teaching through lockdown did. Ensuring wellbeing needs are met in “normal” times may help increase resilience when novel challenges arise.

Williamson, J., Suto, I., Little, J., Jellis, C., & Carroll, M. (2021) Learning during lockdown: How socially interactive were secondary school students in England? Research Matters: A Cambridge University Press & Assessment publication, 32, 22-44.

For countless students, national lockdowns in 2020 and 2021 caused serious upheaval in their education. Across England, decisions to close schools engendered much anxiety, as did Government expectations that most students continued their schooling from home. In addition to lost opportunities for learning and even loss of learning, students’ wellbeing was a significant concern for parents, teachers, and other stakeholders.

Students’ social interactions with their teachers, each other, family and friends are critical to both pedagogy and wellbeing. We report on a survey of over 600 secondary school students’ perceptions of the extent and nature of such interactions during England’s national lockdown in early 2021. We found that the activity types that occurred both within and outside of lockdown schooling changed markedly compared with during pre-pandemic schooling. Students reported spending less time interacting with their teachers and peers though whole class work, small group work, and pair work, and more time working independently. Over half of the students surveyed perceived working independently to be helpful or really helpful, apparently valuing the autonomy they had gained. Patterns of activity types for students who learned mostly or entirely at home were strikingly like those of students who continued to attend school during lockdown; the nature of face-to-face schooling appeared to have changed temporarily in the direction of remote schooling.

• Foreword Tim Oates
• Editorial Tom Bramley
• Learning during lockdown: How socially interactive were secondary school students in England? Joanna Williamson, Irenka Suto, John Little, Chris Jellis, Matthew Carroll
• How well do we understand wellbeing? Teachers’ experiences in an extraordinary educational era Chris Jellis, Joanna Williamson, Irenka Suto
• What do we mean by question paper error? An analysis of criteria and working definitions Nicky Rushton, Sylvia Vitello, Irenka Suto
• Item response theory, computer adaptive testing and the risk of self-deception Tom Benton
• Research News Anouk Peigne

Chambers, L., Williamson, J. and Child, S. (2019). Moderating artwork: Investigating judgements and cognitive processes.  Research Matters: A Cambridge Assessment publication, 27, 19-25.

In this article, we explore the cognitive process and resources drawn upon when moderating artwork. The cognitive processes involved in the external moderation of non-exam assessments has received little attention; the few research studies that exist investigated moderation where the candidates’ submissions were in mostly written form. No studies were found which explicitly looked at a non-written submission, such as artwork. In this small- scale study, participating moderators were asked to “think aloud” whilst moderating candidates’ Art and Design submissions. An analysis of the resulting verbal protocol and observational data enabled timelines of moderator activity to be produced. From these, a process map containing moderation stages, activities, cognitive processes, and resource use was developed.

Williamson, J. (2018). Characteristics, uses and rationales of mark-based and grade-based assessment. Research Matters: A Cambridge Assessment publication, 26, 15-21.

Mark-based assessment requires assessors to assign numerical marks to candidates’ work, assisted by a mark scheme. In grade-based approaches, assessors evaluate candidates’ work against grading criteria to decide upon a grade, avoiding marks altogether. This article outlines the characteristics, uses and rationales of the two approaches, focusing particularly on their suitability for assessment in vocational and technical qualifications.

Williamson, J. and Carroll, M. (2018). Examining the use of technical qualifications within Key Stage 5 programmes of study. Presented at the 5th International Conference on Employer Engagement in Education and Training, London, UK, 5-6 July 2018.

The conference proceedings can be viewed here.

Vitello, S. and Williamson, J. (2018). Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment.

Vidal Rodeiro, C.L. and Williamson, J. (2017). Presented at the European Conference of Educational Research, Copenhagen, Denmark, 22-25 August 2017.

Williamson, J. (2016). Statistical moderation of school-based assessment in GCSEs. Research Matters: A Cambridge Assessment publication, 22, 30-36.

Moderation of school-based assessment (SBA), such as coursework, is required in order to ensure the comparability of marks across different centres. Under current procedures for GCSEs, moderators re-mark a sample of SBA from each centre in order to check whether any adjustment should be made to that centre’s marks. This article explores statistical moderation, an alternative form of moderation that calibrates SBA marks on the basis of a statistical relationship with another assessment, such as an exam component. The article outlines methods of statistical moderation that are used in jurisdictions around the world, and explores the effect of applying these methods to results data from three GCSEs. The analysis focuses on comparing the statistically moderated results to operational results (moderated under existing, non-statistical procedures) in terms of marks, grades, and the rank-order of candidates and centres.