Grading is the procedure of setting thresholds, boundaries or cut-off scores to distinguish how well a candidate has performed in assessments for a particular syllabus. For these grades to be valid, we need to make sure that they are reliable and can be compared across subjects and years.
In her upcoming workshop, Dr Ellen Leggate, Senior Assessment Advisor at Cambridge Assessment International Education will cover some of the key approaches for setting and maintaining a grading standard and the sources of evidence which can be used.
In the latest blog post from The Assessment Network, Ellen, drawing from her career experience, provides insights into the world of grading standards. This includes explanations of the processes involved, highlights of recent developments, and the fundamental principles underpinning the subject.
Could you describe the process of setting and maintaining a grading standard?
Put most simply, it means determining, for each grade, the lowest mark for which performance can be deemed equivalent to that shown at the lowest mark for the same grade in the previous year. In practice, achieving this is often not straightforward! Frequently, there are multiple, conflicting sources of evidence which need to be evaluated, and there is more than one legitimate approach to answering the question of where the grading standard should be set.
What are some developments in this topic which particularly excite you?
The research division at Cambridge University Press & Assessment are investigating whether we can use the marks from all the assessments taken by all candidates as they emerge during an exam series to establish a candidate ability scale prior to the grading process.
This would be a powerful use of our big data sets and a valuable additional strand of evidence to support the setting of grade thresholds for each individual syllabus. While the research theory is strong, it’s a very different matter to take an idea and operationalise it at scale though – particularly given the time pressures between exams being marked and graded!
In your view, what are the key principles of setting and maintaining a grading standard?
The key principles are understanding the variables which affect the outcomes of the assessment.
The two fundamental variables which we will look at in detail in the workshop are: assessment difficulty and cohort ability. Participants will quickly come to appreciate that these two variables are very difficult to disentangle – during the grading process we frequently need to ask questions such as, ‘have outcomes improved because the test was easier this year or because the candidates are more able?’. In many cases, a combination of both factors is responsible.
In the workshop we will consider the factors which contribute to assessment difficulty, such as the content and skills requirements specified in the syllabus, the kind of test and its features, and the approach to marking or moderation. We will also look at what kinds of evidence you can use to evaluate cohort ability.
Why is setting and maintaining a grading standard an important skill for an assessment practitioner to develop?
Grading standards are important for fairness and to make sure that assessment results hold their meaning over time. In your day-to-day practice you may set your students tests which vary in difficulty. By grading the tests, you can put the results on a common scale. This allows the results to be compared in a meaningful way between tests and so gives them currency for users and wider stakeholders. In other words, using a grading scale allows us to have more confidence and draw more meaningful conclusions from what the results from different assessments show.
In the workshop we will consider how grades are used as a basis to interpret and compare assessment results. You will have the opportunity to engage in discussions about how this is relevant to your context.
Why should someone attend your upcoming assessment practitioner workshop?
You should attend the assessment practitioner workshop to find out more about why maintaining a grading standard is important; what the factors are which can make grading a challenging process in practice; and what approaches to grading could be appropriate for your context.
As part of the workshop, you will have the opportunity to engage in discussions, and to work with a group to carry out a grading activity.
What is your background in assessment with a particular focus on setting and maintaining a grading standard?
I work in the Assessment Standards and Quality team at Cambridge International Education. A key part of my role is reviewing and confirming the grade threshold decisions made by colleagues in the assessment division every series for high-stakes international qualifications (IGCSE, international O Level and A Level). My team scrutinises all of the grading decisions to make sure that they are accuracy, consistent and credible.
I have previously worked in scientific research, as a classroom teacher and as an assessment manager for Cambridge International Education and so bring a background in analysis, education and assessment to my current role.
You can join the upcoming workshop: Setting and maintaining a grading standards which is led by Dr Ellen Leggate in collaboration with The Assessment Network.
The Assessment Network is part of Cambridge University Press & Assessment. We provide professional development for impactful assessment.