||27 Jan 2021
This interactive workshop will provide an overview of the history, theory and practice of validation whilst giving consideration to the various mixed messages, controversies and shifting emphases that have been favoured over time. There will also be an opportunity to explore validation methodologies and discuss challenges to their implementation.
The workshop will be relevant to those working in an assessment related role, or anyone with an academic interest in assessment and the theory of validity. This includes teachers, assessors, and key practitioners working for assessment and examination bodies.
If you are new to a theoretical understanding of validity, or have limited experience, this workshop will give a comprehensive overview of the key academic and philosophical considerations. If you are more experienced in the field, this workshop offers an opportunity to refresh your knowledge and engage in debate.
The workshop will explore the relationship between validity and validation with reference to disparate views (or validity/validation ‘camps’). It will also consider how definitional ambiguities affect validation practice.
Contemporary theoretical perspectives will be explored, including the guidance provided by seminal documents such as the US Standards for Educational and Psychological Testing and the argument-based approach to validation championed by Michael Kane. Two philosophical viewpoints, validation as scientific inquiry, and an argument-based approach to validation will be unpacked, then critiqued.
We will also explore validation methodologies and discuss challenges to their implementation:
- What counts as validation evidence?
- How little validation evidence:
- Can we rely upon?
- Should we rely upon?
- Do we rely upon?
Key learning outcomes
By the end of the workshop you will have:
- Gained an appreciation of the contested origins of validation theory
- Developed knowledge of the theory underpinning validation endeavours
- An understanding of how to conduct a validation study
- An awareness of some of the challenges and limitations of undertaking a validation study
Stuart Shaw began his career as an engineer, and holds an honours degree in Physics, a diploma in Applied Physics and a research degree in Metallurgy. His early experience, gained with an international plc, covered a range of engineering specialisms. Following his time in industry, he entered the TEFL world (Teaching English as a Foreign Language), gaining a certificate and diploma in TESOL and a Master degree in Applied Linguistics. He had several years of experience as an EFL teacher and Director of Studies. Stuart also holds a postgraduate degree in Theology. Stuart has worked for Cambridge Assessment since January 2001 where he is particularly interested in demonstrating how Cambridge Assessment seeks to meet the demands of validity in its assessments. Before leading a research team in the area of mainstream international examinations, Stuart worked on a range of Cambridge English products with specific skill responsibilities for assessing writing. He is an experienced presenter and has lectured for the Department of Theoretical and Applied Linguistics (University of Cambridge). He also lectures at the Institute of Education, University of London.
He has a wide range of publications in English second language assessment and educational research journals. Recent assessment books include: Examining Writing: Research and practice in assessing second language writing (Shaw & Weir, 2007); The IELTS Writing Assessment Revision Project: towards a revised rating scale (Shaw & Falvey, 2008); Validity in Educational and Psychological Assessment (Newton & Shaw, 2014); and Language Rich: Insights from Multilingual Schools (Shaw, Imam & Hughes, 2015). He is currently co-authoring a book entitled Is Assessment Fair? (to be published by SAGE in 2020). He has contributed to several international debates on validity (e.g. through Special Issue of Assessment in Education: Principles, Policy & Practice, 2016). He is a Fellow of the Association for Educational Assessment in Europe (AEA-Europe) and Chair of the Scientific Programme Committee. He is also a Fellow of the Chartered Institute of Educational Assessors (CIEA).