-
Research Matters 21 - Foreword
Oates, T. (2016). Foreword. Research Matters: A Cambridge Assessment publication, 21, 1.
Those reading this issue of Research Matters should not tire of reading before they reach the final article by Bramley andWilson.And of course, this is not a cue to skip every other article. Bramley andWilson have continued assiduously to examine the assets and limitations of expert judgement – this article on judging item difficulty accompanies continuing work on the comparison between ‘statistics’ and ‘judgement’ in awarding.
Download
-
Research Matters 21 - Editorial
Green, S. (2016). Editorial. Research Matters: A Cambridge Assessment publication, 21, 1.
The first four articles in this issue report on a range of studies in Higher Education (HE) contexts. Johnson’s article takes us in a different direction as he reports on a study into the way in which senior examiners communicate with their team members in a digital environment. Research by Hughes and Shaw seeks to identify and explain the problem of under-used marks. In the final article, Bramley and Wilson describe two methods designed to support the maintenance of test standards.
Download
-
The effect of specialism and attainment in secondary school on the choice of Higher Education institution and field of study
Sutch, T., Zanini, N. and Vidal Rodeiro, C.L. (2016). The effect of specialism and attainment in secondary school on the choice of Higher Education institution and field of study. Research Matters: A Cambridge Assessment publication, 21, 2-11.
Progression from secondary to Higher Education (HE) has direct implications on wage returns and social mobility. The recent expansion of HE in the majority of European countries has highlighted that returns to HE are not just associated with the decision to study at university rather than enter the labour market, but also with the choice of studying in a particular field at a specific HE institution. Because the process of application and admission to universities in the United Kingdom (UK) places a strong weight on attainment, both overall and in specific subjects, the educational background of students is a key factor influencing progression from secondary education to specific fields of study and HE institutions. The aim of this article is to provide evidence about the relationship between educational background, measured by subject choice and attainment in the final years of secondary education,and HE participation in terms of institution attended and choice of the field of study, an area in which not much research has been carried out so far.
Download
-
The Mathematics needs of prospective Architecture undergraduates
Darlington, E. and Bowyer, J. (2016). The Mathematics needs of prospective Architecture undergraduates. Research Matters: A Cambridge Assessment publication, 21, 11-16.
This article concerns the findings of a large-scale study by the authors of the maths needs of over 4,000 prospective undergraduates of STEMM and Social Science degrees. It reports the responses from Architecture students to an online survey containing questions about the Maths they studied at A-level, how well they perceived the A-level to be as preparation for the mathematical component of their degree, and ways in which they believe A-level Maths could be improved to suit the needs of future prospective undergraduates in their field. Generally, Architecture undergraduates who had taken A-level Maths were positive about it as preparation for their degree, with Mechanics units perceived as the most useful to study if preparing for tertiary Architecture.
Download
-
Assessing the equivalencies of the UCAS tariff for different qualifications
Gill, T. (2016). Assessing the equivalencies of the UCAS tariff for different qualifications. Research Matters: A Cambridge Assessment publication, 21, 16-23.
In the United Kingdom (UK) the Universities and College Admissions Service (UCAS) provides the application process for most universities. The UCAS tariff points system is used by universities to help them select students for their courses. Each grade in eligible qualifications is allocated a points score,which can then be summed in order to provide an overall points score for each student. The allocation of points is such that,in theory,students with the same overall points score gained from different qualifications can be considered to be of equivalent ability or potential. The purpose of this article is to test whether this assumption works in practice, by calculating empirical equivalencies of the UCAS tariff for different qualification.
Download
-
Taking risks and being creative: Assessment in Drama and Theatre
Carroll, P. and Dodds, E. (2016). Taking risks and being creative: Assessment in Drama and Theatre. Research Matters: A Cambridge Assessment publication, 21, 23-28.
In this article we discuss the concept of creativity and its assessment. Creativity is critical to many subjects in secondary education, including Drama and Theatre, but is not easy to assess. Whilst there is a need for reliable assessments at General Certificate of Education (GCE) Advanced Level (A level), the validity and integrity of what is taught are also essential. We describe a small-scale study in which four course leaders at Higher Education Institutions (HEIs) were interviewed about Drama at undergraduate level. The aims of this study were to gain an insight into undergraduate assessment practices, and to identify any lessons to be learned and applied at A level.
Download
-
All in good time: Influences on team leaders’ communication choices when giving feedback to examiners
Johnson, M. (2016). All in good time: Influences on team leaders’ communication choices when giving feedback to examiners. Research Matters: A Cambridge Assessment publication, 21, 28-33.
During the standardisation and live marking phases of an examination session it is increasingly common for team leaders and examiners to interact via a digital marking environment. This environment supports a number of quality assurance functions. Team leaders can see examiners’ real time scripts and mark submissions, and they can compare examiners’ marks with preordained definitive marks on special monitoring scripts to check marking accuracy. The digital marking system also allows team leaders to give examiners feedback on their marking. This article focuses on feedback practices, using observation, interview and survey data from 22 team leaders and six examiners to explore the rationales and communication choices involved in such practices. The analyses suggest that the objective of giving feedback is to construct messages that allow examiners insights into a team leader’s thinking, and this interaction is central to the development of an examiner’s understanding of how to interpret and apply mark schemes in accordance with their team leader’s views. This article discusses how the concept of synchrony underpins the feedback practices of team leaders and examiners, ensuring that the participants develop a shared focus so that both feedback message intention and reception are aligned.
Download
-
Not dumbing down but stimulating up: Reading in the reformed GCSE Modern Foreign Languages classroom
Wilson, F., Smith, K. and Werno, M. (2016). Not dumbing down but stimulating up: Reading in the reformed GCSE Modern Foreign Languages classroom. Research Matters: A Cambridge Assessment publication, 21, 34-41.
Modern Foreign Language (MFL) skills in England have become a cause for concern in part due to declining uptake and low employer satisfaction with students' foreign language skills. The current programme of reform for GCSEs and A levels in MFL aims to address these issues. In this article we first provide a brief overview of some psycholinguistic underpinnings of reading in a foreign language, and then relate this to the context of learning a language at GCSE and A level. We then present the findings from a focus group of MFL teachers which explored the use of reading resources at GCSE with respect to the reformed GCSE curriculum.
Download
-
Why do so few candidates score 4 out of 8 on this question? The issue of under-used marks in levels-based mark schemes
Hughes, S. and Shaw, S. (2016). Why do so few candidates score 4 out of 8 on this question? The issue of under-used marks in levels-based mark schemes. Research Matters: A Cambridge Assessment publication, 21, 42-48.
Marks on a question which are rarely achieved by students are ‘dead marks’ or ‘under-used’ marks. Under-used marks may have a detrimental effect on reliability and can reduce the discriminative powers of a test. It is necessary to ensure, therefore, that there are as few under-used marks as possible. This study aimed to: identify under-used marks in a History examination for 16 year olds; explain this occurrence; and present recommendations to ensure that under-used marks are minimalised. Some under-used marks were identified. Interviews with markers explained the under-use of marks in terms of three issues: the specific skills being assessed, particularly the progression from description to explanation; features of the questions, including the predictability of question structure; and the familiarity of the topic being assessed. The research highlighted some concerns which were already articulated by senior examiners and which resulted in a re-designed mark scheme which aims to support examiners judging the quality of answers.
Download
-
Maintaining test standards by expert judgement of item difficulty
Bramley, T. and Wilson, F. (2016). Maintaining test standards by expert judgement of item difficulty. Research Matters: A Cambridge Assessment publication, 21, 48-54.
This article describes two methods for using expert judgments about test items to arrive at a cut-score (grade boundary) on a new test where none of the items has been pre-tested. The first method required experts to estimate the mean score on the new items from examinees at the cut-score, basing their judgments on statistics from items judged to be similar on previous tests. The second method only required them to identify previous items that they deemed effectively identical in terms of difficulty. Both methods were applied to an AS Chemistry unit. Both methods gave results close to the actual cut-scores, but with the first method this may have been fortuitous since there were quite large differences between the judges’ individual results. The results from the second method were quite stable when the criteria for defining effectively identical items were varied, suggesting this method may be more suitable in practice.
Download
-
Research News
Barden, K. (2016). Research News. Research Matters: A Cambridge Assessment publication, 21, 54-55.
A summary of recent conferences and seminars, and research articles published since the last issue of Research Matters.
Download
-
Statistical Reports
The Research Division. (2016). Statistical Reports. Research Matters: A Cambridge Assessment publication, 21, 56.
The ongoing Statistics Reports Series provides statistical summaries of various aspects of the English examination system, such as trends in pupil uptake and attainment, qualifications choice, subject combinations and subject provision at school. This article contains a summary of the most recent additions to this series.
Download