We need research! Bringing research insights to our agile digital innovation team

Last year Sarah Hughes, Research and Thought Leadership Lead at Cambridge University Press & Assessment, wrote about how research practices needed to fit within 'agile' ways of working. Sarah suggested that ‘agile researchers’ may have to streamline traditional (academic) research processes so that researchers can provide insights to product teams in a timely and more useable manner.

In this blog, I’m going to share my personal experience as a researcher within an agile working environment, and use this as an opportunity to reflect more deeply on Sarah’s view of ‘agile research’. How did it work for me? What went well? What was challenging?

What research support did the agile team need?

The agile team was a newly formed product innovation team, set up to develop digital high-stakes assessments. I was asked to work specifically with the History sub-team, who were at the very early stages of their product journey – the ideas stage. Conversations with History teachers and Higher Education (e.g. History undergraduate courses) highlighted that their learners had gaps in research skills. Therefore, the team was particularly keen to explore the idea of assessing research skills in a more authentic way.

Now, how could they translate that idea into a useful and effective educational product? That was the key question, and where I came in. My role was to show the team what the academic and educational literature could tell us about the pedagogy and assessment of research skills. Reflecting on my time, I can see that I broadly followed this 4-step process.

Defining questions

In total, I worked on a set of three research questions for the design team. Each one was about an aspect of assessment, teaching, and/or learning that the team needed evidence on to make decisions. Importantly, the team were open-minded about what the evidence might show. And the wording of the research questions reflected this – they were all open-ended and non-directional.

For each question, I had to make sure I understood what it meant and why it was being asked.

• The question – did my interpretation of the question match the team's?
• The context – e.g., which educational system(s), school stage(s), curriculum(a) did this apply to?
• The need – what was the team's information gap? What didn't they know? Why did they need to know about this?
• The use – how was the team going to use the information?

Gathering literature

Gathering the literature was the next step. But, it was never as simple as it first seemed. There were various research obstacles in my quest for answers.

• There was often little direct research within the academic literature (i.e., peer-reviewed).
• There was a large (sometimes overwhelming) amount of potentially-relevant information from outside academics – e.g., from teacher websites, blogs, or textbooks.
• There was no single place that contained all the literature I might need.

Analysing and synthesising

Within my research process, there was not a chronological separation between this step and the gathering of literature. They were interdependent and iterative.

• I analysed and synthesised the information as I went along.
• Sometimes my analysis of findings propelled me back into the literature to search for further evidence.

A key decision in this stage was how to collate all the information together. I chose to use Microsoft Excel as my tool. Excel is a much more versatile software than we often give it credit for. I regularly use it to keep track of written pieces of information, and to record and organise my thoughts on them. During this History work, I ended up with several Excel spreadsheets.

• I used the rows to separate different pieces of information or evidence e.g., individual points in research papers, teaching methods, and assessment methods).
• I used the columns to keep notes of key features or thoughts about each piece of information.
• This became a database that I could use to compare and contrast the different pieces of information.
• I could also re-organise information and even convert some of them into numerical data to produce graphs to visualise how common different findings were.

In addition, the analysis and synthesis process reminded me that, when it comes to education, there is (almost) never a single interpretation or conclusion to research findings.

• There is always more information that might exist in the literature that could provide us with more insights.
• There is always room for more primary research.
• What is important is getting to a point where we are confident that our research insights are informed, reasoned, and going to be useful. And, equally important, these insights need to have an audit trail, which anyone can use to trace them back to the source.

Reporting back to the team

It was clear from the start that the product team wanted a quick turnaround of research insights.

• They did not have time for us to write up detailed research reports, and they didn’t need all the information we’d typically include.
• The short timeframes (usually 2 weeks) meant that the findings were usually preliminary, rather than final. As the researcher, I felt it was my responsibility to make sure the team viewed the findings within the context of how I conducted the research.

How could I bring the research insights to the team in a way that ensured they had all the key details but also understood the research caveats?

We achieved this by having face-to-face meetings, which were devoted to talking through the findings together. I liked this approach for many reasons.

• It enabled us to talk openly and honestly about the research.
• It gave me the opportunity to reiterate caveats.
• It allowed us, as a team, to positively interrogate the research findings, working together to pull them apart and put them back together again to arrive at research insights that were useful for the team and their decision-making.
• In many cases, my interpretations changed and evolved as part of those discussions!

After the meetings, I would also give the team some of the outputs I’d produced during the analysis/synthesis stage.

• These outputs were informal, and I’d usually have to tweak or re-organise the information a little, to ensure that someone (other than me) could follow the content.
• I ensured that they contained information and links to all the research I’d reviewed. That way the team could look up the evidence themselves too.

What is agile research to me?

Blog by Sylvia Vitello, Senior Research Officer

If you would like to learn more about the work of our research team, and the development of new digital high-stakes assessments, you can sign up to join our seminar, Developing Research Informed Digital Assessments on 24 November.