While investigating design challenges for national assessment in this 'accountability era', Sandra Johnson explores international performance surveys which pre-date PISA.
When international assessment programmes are mentioned – or indeed, any international comparison of educational performance – it’s PISA which tends to dominate discussion. It shouldn’t. Sandra Johnson’s wide-ranging paper is a timely reminder that earlier surveys existed and regional and country activity both pre-dated PISA and continues to contribute to our understanding of what is happening around the world. Her thoughtful paper presents a penetrating analysis of the history and context of the different programmes, as well as a reflection on the measurement practices.
For all the discussion of where lie the boundaries of ‘validity’ in assessment, Sandra – rightly in my view – recognises the critical importance of ‘washback’ from any form of national or international assessment. Accountability and assessment now walk hand-in-hand, and interact in a negative or supportive way, depending on a complex mix of cultural, political and practical factors. I like very much her discussion of lesser-known periods of history and countries which have not attracted much headline attention. The exploration of Switzerland is particularly insightful – a country about which assumptions abound but on which accurate stories are scarce.
One can argue on technical grounds (and some have) that wider impact should not be seen as part of the ‘validity’ of an assessment – but if this results in closing one’s ears and eyes to the wider impact of assessment, then I think the approach is naïve. The possibility of harnessing accountability policy for desirable policy aims, and of avoiding unintended consequences, lies in sophisticated management of the link between assessment and accountability. And accountability, just like assessment, can take many forms. I have written elsewhere about how Finnish schools feel extremely accountable to their communities, and both test a lot in primary (particularly to identify those pupils at risk of falling behind), and give plenty of assessment data to the local municipality. Just because accountability in Finland assumes a different form to accountability in the US or England, does not mean it is not present, felt, and active in improving education. Sandra’s excellent comparisons over time and between places suggests that there are plenty of policy choices to be made about the form and scheduling of assessment, even if there appears to be plenty of examples of a move towards greater use of high(er)-stakes assessment and accountability arrangements.
What could be a monstrous 300-page report is actually an insightful, wide-ranging and highly readable account of ‘national assessment in this accountability era’. And it’s not something which ignores complexity and detail, but offers policy-makers, researchers and commentators great insight into an area of existing but escalating importance.
Tim Oates CBE
Director of Assessment Research and Development, Cambridge Assessment
Read Sandra's full paper 'Design challenges for national assessment in this accountability era'.