||Stuart Shaw (Principal Research Officer, Cambridge Assessment International Education)
||27 Feb 2020
The Triangle Building, Shaftesbury Road
This session will respond to two fundamental questions about validity in assessment:
- What does it mean to claim validity?
- How can a claim to validity be proven?
Stuart will begin by tracing the evolution of validity from disparate and contested origins to a point where there now appears to be a strong professional consensus over its meaning - reflected in the maxim ‘all validity is construct validity’. Despite fairly general professional agreement over some of the fundamental principles of validity, there still remains considerable confusion over the scope of the concept. Even the consensus over the construct validity maxim seems problematic when scrutinised carefully.
Discontent with the broad perspective has been voiced by a number of notable validity theorists. In this workshop you'll be given an opportunity to explore issues surrounding the current consensus as well as the chance to consider alternate, divergent conceptions of validity.
The presenter approached a vast topic with a critical and practical goal and succeeded in giving participants something firm and concrete regarding theoretical viewpoints as well as laying out their differences and their weakness."
Stuart will highlight the challenges faced when validating the intended interpretation of test scores and their relevance to the proposed uses of those scores.
Key learning outcomes
Specifically, he'll address a number of outstanding validation challenges:
- Appreciation of the lexical and conceptual ambiguities surrounding validity
- Understanding of the different ‘senses’ of validity
- Appreciate of how the concept of validity has evolved over time
- Identifying and reducing the major threats to validity
- Reflection on how the future of validity thinking may develop
This session is aimed at practitioners in education and people working in educational and vocational assessment. This includes teachers, anyone who is directly involved in curriculum planning and materials development, key practitioners in assessment and examination bodies and those with an academic interest in assessment.
Resources are available for anyone who is unfamiliar with the concept of validity, and who may benefit from reading one or two articles prior to the workshop.
There is also literature available for those who are already engaged with the issues of validity theory and validation practice and for those who wish to explore the technical and philosophical foundations of validity in more detail.
Stuart Shaw began his career as an engineer, and holds an honours degree in Physics, a diploma in Applied Physics and a research degree in Metallurgy. His early experience, gained with an international plc, covered a range of engineering specialisms. Following his time in industry, he entered the TEFL world (Teaching English as a Foreign Language), gaining a certificate and diploma in TESOL and a Master degree in Applied Linguistics. He had several years of experience as an EFL teacher and Director of Studies. Stuart also holds a postgraduate degree in Theology. Stuart has worked for Cambridge Assessment since January 2001 where he is particularly interested in demonstrating how Cambridge Assessment seeks to meet the demands of validity in its assessments. Before leading a research team in the area of mainstream international examinations, Stuart worked on a range of Cambridge English products with specific skill responsibilities for assessing writing. He is an experienced presenter and has lectured for the Department of Theoretical and Applied Linguistics (University of Cambridge). He also lectures at the Institute of Education, University of London.
He has a wide range of publications in English second language assessment and educational research journals. Recent assessment books include: Examining Writing: Research and practice in assessing second language writing (Shaw & Weir, 2007); The IELTS Writing Assessment Revision Project: towards a revised rating scale (Shaw & Falvey, 2008); Validity in Educational and Psychological Assessment (Newton & Shaw, 2014); and Language Rich: Insights from Multilingual Schools (Shaw, Imam & Hughes, 2015). He is currently co-authoring a book entitled Is Assessment Fair? (to be published by SAGE in 2020). He has contributed to several international debates on validity (e.g. through Special Issue of Assessment in Education: Principles, Policy & Practice, 2016). He is a Fellow of the Association for Educational Assessment in Europe (AEA-Europe) and Chair of the Scientific Programme Committee. He is also a Fellow of the Chartered Institute of Educational Assessors (CIEA).