-
Research Matters 34 - Foreword
Oates, T. (2022). Foreword. Research Matters: A Cambridge University Press & Assessment publication, 34, 4.
“Lest we forget.” As a nation, England in 2020 ramped up its research on young people and the formal data requirements of schools – ranging from interviews and surveys of pupils and teachers to national submission of data on pupil absences. The findings of all studies in England converge on a single view of the disruption – while a few children benefited from the processes of remote learning and time in the home, the pattern of impact for the vast majority of the children in the country is negative, highly individualised and variable. It cannot be a “return to normal” in our support for any of these affected groups; they require us not to forget them. We need to research them for the purpose of action – to understand their circumstances and individual and collective experiences, and to put in place effective evidence-based support. And quickly.
Download
-
Research Matters 34 - Editorial
Bramley, T. (2022). Editorial. Research Matters: A Cambridge University Press & Assessment publication, 34, 5.
The first article in this issue is another contribution to the large amount of research on the impact of the COVID-19 pandemic on education, as perceived by teachers. Our second article is more technical but right at the heart of assessment: how to maintain or link standards from one version of a test or exam to another. Our third article is an interesting exploration of a large data set from Cambridge CEM’s BASE assessment. Our fourth article reflects on the concept of “recovery curricula” developed in response to educational disruption. The final article is a bit of a departure from our usual fare. We investigated whether there are any systematic differences in the exam results of groups of students with different categories of surname and found a small effect in line with our hypothesis: average grades of candidates with “occupational” surnames were slightly lower than those in other categories.
Download
-
Learning loss in the Covid-19 pandemic: teachers’ views on the nature and extent of loss
Carroll, M., & Constantinou, F. (2022). Learning loss in the Covid-19 pandemic: teachers’ views on the nature and extent of loss. Research Matters: A Cambridge University Press & Assessment publication, 34, 6–25.
The Covid-19 pandemic caused unprecedented disruption to education around the world. As education systems gradually return to normal, there is a push to understand effects of the disruption. A major impact on students is “learning loss”, in which attainment and progress may have fallen behind expected levels. Various efforts have been made to quantify learning loss, but to better understand it, further work, combining quantitative and qualitative approaches, is required.
Here, we sought to record teachers’ views on how far behind (or ahead) their students were compared to a “typical” year, and to gather their opinions about what had been lost (or gained). To do this, we surveyed teachers in schools that work with Cambridge CEM. We received over 400 responses, spread across 38 countries and 198 schools, thus giving a broad sample of experiences.
A majority of respondents felt their students were behind expectations. 1–2 months behind was the most common estimate, but some respondents made much larger estimates of loss, while a sizeable minority thought that their students were on track or even ahead of expectations. Descriptions of the areas of loss indicated that fundamental literacy and numeracy skills had been affected, as had practical skills and general study skills. Responses also described variable impacts, both within and between groups of students. Effects of Covid-related disruption on education are ongoing and may be felt for some time still to come. By exploring the nature and extent of learning loss in students, it is hoped that it will be possible to better understand, and hopefully mitigate, these longer-term impacts.
Download
-
Which assessment is harder? Some limits of statistical linking.
Benton, T., & Williamson, J. (2022). Which assessment is harder? Some limits of statistical linking. Research Matters: A Cambridge University Press & Assessment publication, 34, 26–41.
Equating methods are designed to adjust between alternate versions of assessments targeting the same content at the same level, with the aim that scores from the different versions can be used interchangeably. The statistical processes used in equating have, however, been extended to statistically “link” assessments that differ, such as assessments of the same qualification type that assess different subjects. Despite careful debate on statistical linking in the literature, it can be tempting to apply equating methods and conclude that they have provided a definitive answer on whether a qualification is harder or easier than others.
This article offers a novel demonstration of some limits of statistical equating by exploring how accurately various equating methods were able to equate between identical assessments. To do this, we made use of pairs of live assessments that are “cover sheet” versions of each other, that is, identical assessments with different assessment codes. The results showed that equating errors with real-world impact (e.g., an increase of 5–10 per cent in the proportion of students achieving a grade A) occurred even where equating conditions were apparently favourable. No single method consistently produced more accurate results than the others.
The results emphasise the importance of considering multiple sources of information to make final grade boundary decisions. More broadly, the results are a reminder that if applied uncritically, equating methods can lead to incorrect conclusions about the relative difficulty of assessments.
Download
-
Progress in the first year at school
Jellis, C. (2022). Progress in the first year at school. Research Matters: A Cambridge University Press & Assessment publication, 34, 42–56.
The results of an assessment taken at the start and end of the Reception Year by children in state schools in England over three years were analysed. Over 70 000 children were assessed during this time. The results of the analysis provided evidence of what the average child could do when they started school, and how much progress they made in that first year. Children typically start school with a wide range of skills and experiences and once they are settled into life in school, they make exceptional progress in their first year.
Download
-
What are ‘recovery curricula’ and what do they include? a literature review
Johnson, M. (2022). What are “recovery curricula” and what do they include? A literature review. Research Matters: A Cambridge University Press & Assessment publication, 34, 57–75.
The concept of educational recovery is relevant to many systems, both those that experience some form of sudden disruption as well as those that historically have been prone to disruption. Our involvement in developing a curriculum framework for displaced learners in the Learning Passport project (UNICEF, 2020) made us more aware of the field of Education in Emergencies. An educational emergency is a situation where “man-made or natural disasters destroy, within a short period of time, the usual conditions of life, care and education facilities for children and therefore disrupt, deny, hinder, progress or delay the realisation of the right to education” (Committee on the Rights of the Child, 2008, p. 1). The COVID-19 pandemic has made the concept of emergency and recovery more relevant to even more education systems. The literature review described in this article was carried out to identify what recovery curricula are (e.g., what they seek to achieve, what information they cover, etc.), as well as to consider any evidence for their efficacy. By exploring the recovery curricula literature, we also wanted to consider the extent to which the concept is a singular, generalisable one, or whether it is tied to specific contexts.
Download
-
What’s in a name? Are surnames derived from trades and occupations associated with lower GCSE scores?
Williamson, J., & Bramley, T. (2022). What’s in a name? Are surnames derived from trades and occupations associated with lower GCSE scores? Research Matters: A Cambridge University Press & Assessment publication, 34, 76–97.
In England, there are persistent associations between measures of socio-economic advantage and educational outcomes. Research on the history of names, meanwhile, confirms that surnames in England – like many other countries – were highly socially stratified in their origins. These facts prompted us to wonder whether educational outcomes in England might show variation by surname origin, and specifically, whether surnames with an occupational origin might be associated with slightly lower average GCSE scores than surnames of other origins. Even though surnames do not measure an individual’s socio-economic position, our hypothesis was that in aggregate, the educational outcomes of a group defined in this way might still reflect past social history.
In line with the research hypothesis, the results showed that the mean GCSE scores of candidates with occupational surnames were slightly lower than the mean GCSE scores of candidates with other surnames. The difference in attainment was a similar size to the difference expected between candidates half a year apart in age, and much smaller than the “gap” between male and female candidates. The explanation for the identified effect was beyond the scope of the current research, but surname effect mechanisms proposed in the literature include the psychological (e.g., implicit egotism), sociological and socio-genetic.
Download
-
Research News
Bowett, L. (2022). Research News. Research Matters: A Cambridge University Press & Assessment publication, 34, 98-100.
A summary of recent conferences, reports, blogs and research articles published since the last issue of Research Matters.
Download