||Dr Tom Benton
||30 Apr 2020
The Triangle Building, Shaftesbury Road
This training event will help you to gain a better understanding of the concept of assessment reliability, including how it is conceptualised, calculated, and should be interpreted. In particular, it will focus on ways of calculating the impact of markers and the exact selection of questions within exams upon the scores achieved by candidates.
“It was great to have a well-qualified person in Tom Benton running a training course like this!”
To begin with, we will explore the different ways in which reliability can be conceptualised. Then, using ideas from classical test theory, we will cover the how these concepts translate into practice for calculating marking reliability and some of the most commonly used test reliability coefficients. Finally, we will discuss the different ways in which reliability can be presented, and will invite participants to look at the advantages/disadvantages of each. The session will also explore the factors that may affect the various reliability coefficients.
Key learning outcomes
- Understand the definition of reliability and why it is important.
- Understand how different reliability indices are calculated.
- Understand how to interpret the various indices in practice and the factors that might affect them.
The course is aimed at anyone who needs to understand how to use and interpret reliability coefficients in practice. This may include those who are responsible for the development and quality control of assessments.
Dr Tom Benton has worked in educational statistics for almost 20 years and is Principal Research Officer in Assessment Research and Development Division, Cambridge Assessment. Prior to joining Cambridge Assessment he worked for the National Foundation for Education Research (NFER) as an expert in the field of statistical analysis. His work at NFER included test development, survey research, programme evaluation, benchmarking and international comparisons.
Tom has been closely involved with a number of large-scale national and international surveys of young people as well as the development and standardisation of numerous educational tests. He has co-authored a report on the subject of assessing reliability which has been published by Ofqual (read the report), and has practical experience of developing reliable ways of measuring attitudes and abilities of interest across a wide range of different subject areas including academic ability, community cohesion, self-efficacy, enjoyment of school, self-confidence and political opinions.