Assessment training: Using item response theory in practice

Assessment training: Using item response theory in practice

Speakers: Dr Tom Benton
Date: 25 Feb 2019 Venue: Cambridge Assessment The Triangle Building, Shaftesbury Road Cambridge CB2 8EA
Time: 10:00 - 16:00
Type: Training event Fee: £200.00

This course will provide an introduction into the theory behind item response theory (IRT) and clear examples of when it is useful. It will include practical activities where participants will use the software package R to fit and interpret IRT models themselves as well as experiment with using IRT to help with test construction.

“Fantastic delivery of the course. Very well structured, engaging throughout.”

Workshop outline

The morning will introduce the basic theory of IRT including, in particular, the different IRT models that are available and the situations in which they may have advantages over simpler classical methods of analysing item-level data. Participants will also learn how to fit IRT models using R and how to interpret the results to assess the quality of items. The afternoon will consider two practical applications of IRT: calibration of items taken by different sets of candidates onto a common difficulty scale, and automated test assembly. In both cases the theory behind the IRT approach will be followed by the practical activities where participants will learn how to perform these tasks themselves.

Key learning outcomes

  • Introduction to item response theory (IRT) and the situations in which it may be more useful than simpler, classical approaches to item analysis.
  • Understand some uses of IRT.
  • Undertake practical analysis in R to understand how to fit IRT models and use them for: evaluating item quality, calibrating the difficulties of items taken by different groups of candidates, and test assembly.
  • Utilise the opportunity to engage with an expert in the field.
  • Understand the different types of IRT models and how they can be useful.
  • Gain practical experience of fitting different IRT models and interpreting the results.
  • Understand how IRT can be used to assist in test assembly.

Intended audience

This course is intended for assessment professionals with an interest in empirical methods of assessing item quality as well as those interested in data driven approaches to test assembly. Participants will be expected to have some existing familiarity with classical item statistics (for example, facilities and discriminations). However, prior experience of using R is not a pre-requisite.

Course trainer

Tom Benton - Principal Research OfficerDr Tom Benton has worked in educational statistics for more than 10 years and is Principal Research Officer in Assessment Research and Development Division, Cambridge Assessment. Prior to joining Cambridge Assessment he worked for the National Foundation for Education Research (NFER) as an expert in the field of statistical analysis. His work at NFER included test development, survey research, programme evaluation, benchmarking and international comparisons.

Tom has been closely involved with a number of large-scale national and international surveys of young people as well as the development and standardisation of numerous educational tests. He has co-authored a report on the subject of assessing reliability which has been published by Ofqual (read the report), and has practical experience of developing reliable ways of measuring attitudes and abilities of interest across a wide range of different subject areas including academic ability, community cohesion, self-efficacy, enjoyment of school, self-confidence and political opinions.

The Triangle Building, Shaftesbury Road Cambridge CB2 8EA

Related links

Contact the Network

Please contact us for further information about the Network events and bespoke training.