The Digital High Stakes Assessment Programme at Cambridge University Press & Assessment is developing innovative assessments and using evidence to inform their design. The pandemic has changed practice in education and assessment, arguably making now the best time to consider novel approaches to assessment. We conducted a rapid review of the research literature to investigate the pros and cons of open book, open web (OBOW) assessments.
What is an open book open, web exam?
In an OBOW exam students have access to online or personal resources during the assessment; this may include classroom notes, reference materials, textbooks, or other pertinent course materials and can apply to paper based or on-screen assessments. This style of assessment is not a new idea, the literature goes back decades. The most recent research is often set in Higher Education and in the medical sector (possibly because of the close match between the characteristics of OBOW exams and the tasks required of the medical profession in their work).
The pros and cons of OBOW exams
The literature describes the merits of OBOW exams:
- Less emphasis on memorising and more on problem solving, critical thinking and other higher level thinking skills
- A more engaging experience resulting in higher student motivation
- Real-world problems which relate to future studies or work
- Being less stressful to revise for and so reducing students’ exam anxiety
- An approach to revision which is more focused on applying what has been learnt than on reproducing what has been learnt than in closed book exams.
There are also concerns around the use of OBOW exams:
- A reliance on digital literacy which may be unfair to some learners
- The question types taking longer to answer and so requiring fewer distinct items in a test
- Increased marking workload
- Test security because students have access to the internet.
Other issues around using OBOW assessments related to the lack of familiarity with a new assessment type and that this might lead to learners’ under-performance or challenges for question writers. Adaptation to OBOW exams takes time and involves a move away from traditional methods to more use of technology in teaching as well as changes to approaches to exam preparation. There is plenty of advice around how to write items for OBOW exams; e.g. to think of the task as a mini context to encourage students to think conceptually about a problem and apply learnt skills and techniques or act out a role.
An additional benefit of OBOW exams could be that malpractice is made more challenging and time consuming, for example by different learners taking different versions of a question.
The big potential pay-off with OBOW exams is that a greater range of skills including problem solving and critical thinking can be assessed. A focus on these skills in the classroom could provide a more authentic and better preparation for future learning and work.
OBOW assessments have the potential to address some criticisms of the current assessment system by, for example, including OBOW assessments as one part of an assessment model whilst recognising that learners exist in an environment where they have instant access to a wealth of digital information.
How we are exploring the OBOW approach
In the Digital High Stakes team we are exploring the value of OBOW in our assessments:
- An assessment of Historical research skills in which learners interact with a digital library of historical sources in a learning and assessment platform.
- Computer Science learners are assessed on their ability to write their own code as well as access and adapt existing code in the same way that coders do in industry. When doing an assessed programming project learners have unrestricted access to online resources and cite the resources they have used.
- Learners of Data Literacy work with a real-world dataset that they choose from a digital bank, perform authentic tasks such as cleaning and transforming data, analysis, and visualising data. Self-selection means that learners engage with data that is personally meaningful and relevant. OBOW enables the use of online tools and techniques for tasks.
Perhaps OBOW assessments are best suited to being part of an assessment system which includes a variety of types of assessment. OBOW could be used to assess some curriculum areas or specific learning objectives which are best suited to that type of assessment. While HE and the medical sector have provided evidence on the merits and limitations of OBOW assessments, what is needed is research on OBOW in the context of high stakes school assessments and how these different assessment types can make the assessment experiences even more relevant, authentic, and fair.
Clare Green, Educational researcher and consultant