Tom Bramley reports from the 13th UK Rasch user group as the annual psychometric modelling meeting returns to Cambridge Assessment.
On Thursday 21st March Cambridge Assessment hosted the 13th annual meeting of the UK Rasch users’ group at The Triangle building in Cambridge. ‘What is a Rasch user?’, you may be wondering.
The Danish statistician Georg Rasch invented a class of probabilistic models known as ‘Rasch models’ that relate the probability of success on a test item to the test taker’s ability and the item’s difficulty. The use of these models was promoted by Benjamin Wright in the USA and others who saw them as the way to bring into the social sciences the kind of measurement seen in the physical sciences.
The zeal with which these models were recommended led to considerable controversy in the 1970s and 1980s with those arguing for the special status of Rasch models (as measurement models) on one side, and those arguing they were just one of many possible ways of statistically modelling responses to test items on the other.
In 2006 Peter Tymms at the Centre for Evaluation and Monitoring in Durham had the idea of getting together people in the UK who were working with the Rasch model, and since then there has been a meeting of this ‘user group’ every year. It has always been well supported by Cambridge Assessment because we have since the 1990s used the Rasch model as the basis for scaling our English language tests, and our researchers have also used it in a variety of theoretical and applied psychometric contexts. Cambridge Assessment last hosted the ‘Rasch day’ in 2008 (at Hughes Hall) and we thought that with our arrival in the new Triangle building it was time to host it again!
The aim of the day is to provide an informal forum for sharing work and networking. The Rasch model is also used in healthcare (for example for measuring rehabilitation, and quality of life), so it is interesting for those of us working in educational assessment to exchange ideas and learn from those working in a different sector. This year’s meeting was very well attended with over 70 delegates from universities and other external organisations including Ofqual, AQA and Pearson.
Cambridge Assessment contributed two presentations: Jo Williamson and Matthew Carroll (pictured below) demonstrated a prototype app that could help give meaning to grade scales (such as the new 9-1 GCSE scale) by graphical display of items that candidates at particular boundaries have an 80% chance of getting right (the app allows users to alter the 80% to a value of their choosing); and Tom Benton presented results showing that the gains in efficiency (better reliability) from adaptive tests do not always translate into actual improvements in predictive value when it comes to correlations with scores on other tests.
Feedback from delegates was very positive with particular praise for the presentation by Karl Bang Christensen from the University of Copenhagen on interpreting fit statistics – something every Rasch practitioner has to grapple with! Presentations from most previous Rasch days can be found on the Rasch user group website and those from this year should appear there in due course.
It’s my impression that the UK Rasch user group is not particularly zealous about the measurement properties of the Rasch model, or anxious to re-ignite the debates of 40 years ago. The approach is more pragmatic – trying to find the best tool for the job – and recognising that within the UK any form of statistical psychometric modelling, Rasch or otherwise, is an extremely niche interest! It’s good for the relatively few of us working on it to get together from time to time.