Irenka Suto

Irenka Suto

I studied at the University of Cambridge, obtaining a BA in Natural Sciences (Part 2 Psychology) and a PhD from the Department of Psychiatry. I stayed on to conduct post-doctoral research, publishing a book and various journal articles on financial decision-making and the assessment of mental capacity among people with intellectual disabilities.

I joined Cambridge Assessment in 2004, working initially as a Research Officer, and then as a Senior Research Officer. I spent several years applying and developing judgement and decision-making theory in the context of examination marking and grading. I also developed a new interest in the curricula linked to assessments, exploring concepts of so-called 21st Century skills, research projects for secondary school students, and alternatives to the most popular qualifications.

Since 2012 I have been a Principal Research Officer. I have led several programmes of research to support the reform of A levels and GCSEs, and I currently lead the qualifications development strand of our divisional research programme. I use a combination of quantitative and qualitative research methods, and I remain very interested in the many human judgements and decisions entailed in educational assessment, as made by students, teachers and examiners. In particular, I am currently interested in the causes of profession error and why detection is not always easy.

Outside of work, most of my time is spent with my young and energetic family. I also enjoy baking, playing classical music, and holidaying in mountainous areas.

Publications

2016

Extending educational taxonomies from general to applied education: Can they be used to write and review assessment criteria?
Greatorex, J. and Suto, I. (2016). Paper presented at the 8th Biennial Conference of the European Association for Research in Learning and Instruction (EARLI) SIG 1 - Assessment and Evaluation, Munich, Germany, 24-26 August 2016
Assessing the transition between school and university: Differences in assessment between A level and university in English.
Wilson, F., Child, S., and Suto, I. (2016). Assessing the transition between school and university: Differences in assessment between A level and university in English. Arts and Humanities in Higher Education, 41(1), 1-21.

2014

Course struggle, exam stress, or a fear of the unknown? A study of A level students’ assessment preferences and the reasons behind them
Suto, I., Elliott, G., Rushton, N. and Mehta, S. (2014). Course struggle, exam stress, or a fear of the unknown? A study of A level students’ assessment preferences and the reasons behind them. Educational Futures (ejournal of the British Educational Studies Association), 6(2).
The Cambridge approach to 21st Century skills: definitions, development and dilemmas for assessment
Suto, I. and Eccles, H. (2014). Paper presented at the annual conference of the International Association for Educational Assessment (IAEA), Singapore, 25-30 May 2014.

2013

A comparison of assessment at school and university: More than just increasing demands.
Wilson, F., Child, S. F. J., & Suto, I. (2013). A comparison of assessment at school and university: More than just increasing demands. Paper presented at the European Conference of Educational Research, 2013.
The first year beyond freshers week: An exploration of additional support and assessment practices at university
Child, S., Mehta, S., Wilson, F., Suto, I. and Brown, S. (2013). Paper presented at the British Educational Research Association (BERA) annual conference, University of Sussex, Brighton, UK, 3-5 September 2013.
A level reform: Is the Government in tune with its stakeholders?

Suto, I., Mehta, S., Child, S., Wilson, F. and Jeffrey, E. (2013). A level reform: Is the Government in tune with its stakeholders? Research Matters: A Cambridge Assessment publication, 16, 9-14.

Reformed GCE A levels are on the educational horizon for many students and their teachers. Awarding bodies are in the process of redeveloping their courses and from September 2015, the new syllabuses will be taught in sixth forms across England. In this article, we give a chronological account of the recent developments in Government policy which have fed into these plans. Alongside this account, we describe five studies that we have undertaken within our Higher Education (HE) Engagement research programme. An overarching aim of our research has been to ascertain the views and experiences of stakeholders in schools, colleges and universities on multiple aspects of A level reform.

Thrown in at the deep end? Exploring students', lecturers' and teachers' views on additional support lessons at university
Mehta, S., Child, S. F. J., Suto, I., & Brown, S. (2013) Conference proceedings. The Future of Education. 3rd Conference
21st Century Skills: Ancient, ubiquitous, enigmatic?

Suto, I. (2013). 21st Century Skills: Ancient, ubiquitous, enigmatic? Research Matters: A Cambridge Assessment publication, 15, 2-8.

The understanding and skills needed to compete in today’s global economy are arguably quite different to those upon which 19th and 20th century education systems have traditionally focussed. Life has become much more international, multicultural and inter-connected. Seismic advances have occurred in ICT and in access to it. These have enabled the economies of developed countries, including the UK’s, to shift from a basis of material goods and services to one of information and knowledge. The aim of this article is to explore some of the benefits and risks of building pedagogies and curricula around 21st Century skills. I begin by outlining some conceptualisations of 21st Century skills. I then address the question of how their development in young people can best be supported; I describe recent examples of alternative approaches used in the UK and internationally, including extended projects for sixth-form students. I also start to consider the value placed by stakeholders on the summative assessment of 21st Century skills, and finally, the feasibility of such assessment for test developers.

Independent research at A level: Students’ and teachers’ experiences

Mehta, S., Suto, I., Elliott, G. and Rushton, N. (2013). Independent research at A level: Students’ and teachers’ experiences. Research Matters: A Cambridge Assessment publication, 15, 9-16. 

Our aims were to explore teachers’ and students’ experiences and perspectives of independent research at A level. The study focused Economics, French and Mathematics. It investigated: (i) the extent to which teachers think research and investigative skills can be developed at A level; (ii) the resources and guidance that students use; and (iii) whether subject-specific differences arise. A questionnaire and follow-on interview methodology was used. 47 Mathematics teachers, 24 Economics teachers and 15 French teachers participated. Additionally, 299 Mathematics students, 228 Economics students and 136 French students took part.

About half of the French and Economics teachers were found to assign investigative/research tasks to their students at least once a fortnight. On the other hand, about half of the Mathematics teachers set such tasks less often and a further 40% never set them at all. The frequency with which the teachers set investigation/research tasks as homework/private study showed the same subject-specific differences as the classroom context. The internet was the most frequently listed source that students across all three subjects consulted while engaging in independent research. The interview data shed further light on general and specific internet usage. Overall, the findings explain some of the variation in preparedness of new undergraduates for independent study and research-related tasks at university.

2012

Analyse, Evaluate, Review….How Do Teachers with Differing Subject Specialisms Interpret Common Assessment Vocabulary?
Nadas, R., Suto, I. and Grayson, R. (2012). Paper presented at the European Conference for Educational Research (ECER), University of Cadiz, Spain, 18-21 September 2012.
An exploration of how independent research and project management skills can be developed and assessed among 16 to 19 year olds
Suto, I., Nadas, R. and Chambers, L. (2012). Paper presented at the British Educational Research Association (BERA) conference, Manchester, UK, 4-6 September 2012.
Students’ and teachers’ views and experiences of A Level module re-sits
Gill, T. and Suto, I. (2012) Paper presented at British Educational Research Association (BERA) conference, Manchester, September 2012
Cambridge Assessment HE research: Survey of lecturers - executive summary
Suto, I. (2012). Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment.
Cambridge Assessment HE research: Qualitative investigation - executive summary
Mehta, S., Suto, I. and Brown, S. (2012). Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment.
Starting them young: research and project management opportunities for 16 to 19 year olds

Suto, I. and Nadas, R. (2012). Starting them young: research and project management opportunities for 16 to 19 year olds. Research Matters: A Cambridge Assessment publication, 13, 27-31.

Several educational routes have been developed which entail project work with a specific focus on independent learning and research. In this article we outline some of the options that exist at Level 3, primarily for 16 to 19 year olds (Years 12 and 13) in the UK and internationally. We then conduct a more detailed comparison of two routes: the Extended Project Qualification, and the International Baccalaureate Extended Essay. Many stakeholders may be unaware of the differences in the aims, structure, and scope of these routes. It is important for students and teachers to be conscious of the differences so that they can make informed decisions about what is most suitable for them. End-users such as higher education admissions tutors and employers also need to understand the differences in order to weigh up the experiences and achievements of applicants fairly.

2011

An exploration of the script features that most influence expert judgements in three methods of determining examination grade boundaries.
Suto, I., and Novakovic, N. (2012). An exploration of the script features that most influence expert judgements in three methods of determining examination grade boundaries. Assessment in Education: Principles, Policy and Practice, 19(3), 301-320.
Going beyond the syllabus: A study of A level Mathematics teachers and students.
Suto, I., Elliott, G., Rushton, N., and Mehta, S. (2011) Educational Studies.
What form of feedback most motivates students? A study of teachers' perceptions of the Impact of assessment
Rushton, N., Suto, I., Elliott. and Mehta, S. (2011). Paper presented at the AEA-Europe annual conference, Belfast, November 2011.
Independent research at A level: students' and teachers' experiences
Mehta, S., Suto, I., Elliott, G. and Rushton, N. (2011). Paper presented at the British Educational Research Association annual conference, University of London Institute of Education, September 2011.
Going beyond the syllabus: views from teachers and students of A level mathematics
Suto, I., Elliott, G., Rushton, N. and Mehta, S. (2011). Paper presented at the British Educational Research Association annual conference, University of London Institute of Education, September 2011.
Small is beautiful? An exploration of class size at A level
Rushton, N., Suto, I., Elliott, G. and Mehta, S. (2011). Paper presented at the British Educational Research Association annual conference, University of London Institute of Education, September 2011.
The interrelations of features of questions, mark schemes and examinee responses and their impact upon marker agreement.
Black, B., Suto, I., and Bramley, T. (2011) Assessment in Education: Principles, Policy and Practice (Special Issue), 18, 3, 295-318
Why study Economics? Perspectives from 16 to 19 year old students
Mehta, S., Suto, I., Elliott, G. and Rushton, N. (2013). Paper presented at the International Association for Citizenship, Social and Economics Education annual conference, Bath Spa University, June 2013.
Who should mark what? A study of factors affecting marking accuracy in a biology examination.
Suto, I., Nádas, R., and Bell, J.F. (2011) Research Papers in Education, 26, 1, 21-51
A critical review of some research methods used to explore rater cognition
Suto, I. (2011). Paper presented at the American Educational Research Association (AERA) Annual Conference, New Orleans, LA, USA, 8-12 April 2011.
Why study economics? Perspectives from 16 to 19 year old students
Mehta, S., Suto, I., Elliott, G. and Rushton, N. (2011) Citizenship, Social and Economics Education

2010

Investigating examiners' thinking: using Kelly's Repertory Grid technique to explore cognitive marking strategies
Suto, I. and Nadas, R. (2010) Research in Education, 84, 38-53
Speed isn’t everything: a study of examination marking
Nadas, R. and Suto, I. (2010) Educational Studies, 36, 1, 115-118
When developing and validating assessments, what are the key issues in psychological research?
Suto, I. & Shaw, S. (2010) British Psychological Society Annual Conference, Stratford upon Avon
The interrelations of features of questions, mark schemes and examinee responses and their impact on marker agreement
Suto, I., Bramley, T. & Black, B. (2010) European Conference on Educational Research (ECER), Helsinki.
How well can teachers assess pre-university students’ research reports? Findings from an empirical study
Suto, I. & Shaw, S. (2010) European Conference on Educational Research (ECER), Helsinki.
Practising what we preach: how do awarding bodies make use of psychological Research methods?
Suto, I. (2010) Association for the Teaching of Psychology Annual Conference, Keele University
A tricky task for teachers: assessing pre-university students' research reports

Suto, I. and Shaw, S.  (2010). A tricky task for teachers: assessing pre-university students' research reports. Research Matters: A Cambridge Assessment publication, 10, 10-16.

In the UK and internationally, many students preparing for university are given the challenge of conducting independent research and writing up a report of around 4,000 or 5,000 words. Such research activities provide students with opportunities to investigate a specialist area of study in greater depth, to cross boundaries with an inter-disciplinary enquiry, or to explore a novel non-school subject such as archaeology, cosmology or anthropology. In this study, we explored the feasibility of applying a single mark scheme to research reports covering diverse topics in order to reward generic research skills. Our aim was to investigate the reliability with which teachers can mark diverse research reports, using four different generic assessment objectives. We also investigated teachers’ views in applying generic mark schemes, particularly when marking reports on unfamiliar topics. Our analyses indicated that marking reliability was good, though like almost all qualifications, imperfect. Possible reasons and explanations for marking difficulty related to subject knowledge, the clarity of student thought, and the overall level of student performance. 

2009

What influences moderation and standards maintenance in school-based summative assessment?
Suto, I. and Shiell, H. (2009) Education Journal, 119, 41-43
Creating research programmes to support the development and validation of qualifications
Suto, I. & Shaw, S. (2009) Association for Educational Assessment (AEA) - Europe, Malta
How should grade boundaries be determined in examinations? An exploration of the script features that influence expert judgements
Novakovic, N. & Suto, I. (2009) European Conference on Educational Research (ECER), Vienna
Capturing expert judgement in grading: an examiner's perspective

King, P., Novakovic, N. and Suto, I. (2009). Capturing expert judgement in grading: an examiner's perspective. Research Matters: A Cambridge Assessment publication, 8, 32-33.

There exist several methods of capturing expert judgement which have been used, or could potentially be used, in the process of determining grade boundaries for examinations. In a recent study, we sought to explore the judgements entailed in three such methods: (i) rank ordering, (ii) traditional awarding, and (iii) Thurstone pairs. A key aim was to identify the features of candidates’ scripts that affect the judgements made in each of the three methods. To achieve this, sixty experienced examiners participated in the study. Each made judgements about overall script quality, using each method on a different batch of scripts. Additionally, each examiner completed a research task in which he or she was asked to rate a fourth batch of scripts for a series of features, using rating scales devised by the researchers. Subsequent data analysis entailed relating the judgemental data on script quality to the script feature data. Immediately after taking part in the study, one examiner recorded and offered the Research Division his views and experiences of participation. His perspective is the focus of this article.

Thinking about making the right mark: Using cognitive strategy research to explore examiner training

Suto, I., Greatorex, J. and Nadas, R. (2009). Thinking about making the right mark: Using cognitive strategy research to explore examiner training. Research Matters: A Cambridge Assessment publication, 8, 23-32.

In this article, we draw together research on examiner training and on the nature of the judgements entailed in the marking process. We report new analyses of data from two recent empirical studies, Greatorex and Bell (2008) and Suto and Nadas (2008a), exploring possible relationships between the efficacy of training and the complexity of the cognitive marking strategies apparently needed to mark the examination questions under consideration. In the first study reported in this article, we considered the benefits of three different training procedures for experienced examiners marking AS-level biology questions. In the second study reported here, we explored the effects of a single training procedure on experienced and inexperienced (graduate) examiners marking GCSE mathematics and physics questions. In both studies, it was found that: (i) marking accuracy was better after training than beforehand; and (ii) the effect of training on change in marking accuracy varied across all individual questions. Our hypothesis that training would be more beneficial for apparently more complex cognitive marking strategy questions than for apparently simple cognitive marking strategy questions was upheld for both subjects in Study 2, but not in Study 1.

Investigating examiners’ thinking: using Kelly’s Repertory Grid technique to explore cognitive marking strategies
Suto, I. and Nadas, R. (2009). Paper presented at the 14th International Conference on Thinking. Pages 210-226 in the Conference Proceedings, Kuala Lumpur, 22-26 June 2009.

2008

A Quantitative Analysis of Cognitive Strategy Usage in the Marking of Two GCSE Examinations
Suto, W. M. I. and Greatorex, J. (2008) Assessment in Education: Principles, Policy and Practice, 15, 1, 73-89
What determines GCSE marking accuracy? An exploration of expertise among maths and physics markers
Suto, W.M.I. and Nadas, R. (2008) Research Papers in Education, 23, 4, 477-497
What do GCSE examiners think of ‘thinking aloud’? Findings from an exploratory study
Greatorex, J. and Suto, W.M.I. (2008). What do GCSE examiners think of ‘thinking aloud’? Findings from an exploratory study. Educational Research, 40, 4, 319-331
What attracts judges’ attention? A comparison of three grading methods
Greatorex, J., Novakovic, N. & Suto, I. (2008) International Association for Educational Assessment (IAEA) Conference, Cambridge
Towards a new model of marking accuracy: An investigation of IGCSE biology
Suto, I. & Nadas, R. (2008) International Association for Educational Assessment (IAEA) Conference, Cambridge
Exploring the role of human judgement in examination marking: findings from some empirical studies
Greatorex, J., Suto, I. & Nadas, R. (2008) Association of Language Testers in Europe (ALTE), Cambridge
What goes through an examiner's mind? Using verbal protocols to gain insights into the GCSE marking process
Suto, W. M. I.and Greatorex, J. (2008) British Educational Research Journal, 34, 2, 213-233

2007

An exploration of self-confidence and insight into marking accuracy among GCSE maths and physics markers
Nadas, R. and Suto, I. (2007). Paper presented at the annual conference of the International Association for Educational Research (IAEA), Baku, Azerbaijan, 16-21 September 2007.
The ‘Marking Expertise’ projects: Empirical investigations of some popular assumptions
Suto, I. and Nadas, R. (2007) International Association for Educational Assessment (IAEA) Conference, Azerbaijan
Exploring how the cognitive strategies used to mark examination questions relate to the efficacy of examiner training
Greatorex, J., Nádas, R., Suto, I. and Bell, J F. (2007) European Conference on Educational Research (ECER) Conference, Ghent, Belgium
An exploration of self-confidence and insight into marking accuracy among GCSE maths and physics markers
Nadas, R. and Suto, I. (2007) British Educational Research Association (BERA) Annual Conference
What makes some GCSE examination questions harder to mark than others? An exploration of question features related to marking accuracy
Suto, I. and Nadas, R. (2007) British Educational Research Association (BERA) Annual Conference

2006

What do GCSE examiners think of 'thinking aloud'? Interesting findings from a preliminary study
Suto, I. and Greatorex, J. (2006) British Educational Research Association (BERA) Annual Conference
An empirical exploration of human judgement in the marking of school examinations
Greatorex, J. & Suto, I. (2006) International Association for Educational Assessment (IAEA) Conference, Singapore

2005

What goes through a marker’s mind? Gaining theoretical insights into the A-level and GCSE marking process
Greatorex, J. and Suto, I. (2005). Paper presented at the Association for Educational Assessment (AEA) - Europe, Dublin, Republic of Ireland, 3 November 2005.
International perspectives on assessment in vocational qualifications (ages 14-19): What can we learn from each other?
Suto, I. and Green, S. (2005). Paper presented at the Association for Educational Assessment (AEA) - Europe, Dublin, Republic of Ireland, 3 November 2005.
What goes through an examiner’s mind? Using verbal protocols to gain insights into the GCSE marking process
Suto, I. and Greatorex, J. (2005) British Educational Research Association (BERA) Annual Conference

Research Matters

Research Matters

Research Matters is our free biannual publication which allows us to share our assessment research, in a range of fields, with the wider assessment community.