Research Bytes - Using e-portfolios to capture and assess transversal skills

Using e-portfolios to capture and assess transversal skills: tensions in theory and praxis

08 Dec 2020 (12:06)

In this presentation Cambridge Assessment researcher Emma Walland presents her findings on using e-portfolios to capture and assess transversal skills.

Video transcript

[00:00:07:20] Hello, I'm Emma from Cambridge Assessment, and today I'm going to be talking to you about the results of some research that I conducted with my colleague Stuart Shaw about using portfolios to capture and assess transversal skills, tensions in theory and practice.

[00:00:23:27] Many students today live in a multimedia hypertextual and digitalised world, but often the assessment and learning methods they experience can be somewhat of an anachronism that can be out of step with their technological lives. Therefore, proponents of portfolios call for the use of technological methods to transform teaching, learning and assessment.

[00:00:44:19] Beyond this, assessment methods are more geared toward assessing products rather than processes like reflection and collaboration, and e-portfolios have been put forward as potential tools to assist these processes.

[00:00:57:13] These are seen to be important skills in today's knowledge [00:01:00:00] economy, and the value of promoting lifelong learning is more important than ever.

[00:01:05:10] However, there are major barriers, including the fact that not all students have the same access to technology.

[00:01:13:04] Several different definitions of portfolios exist, but they generally refer to purposeful collections of information and digital artefacts that demonstrate developmental evidence, learning outcomes, skills or competencies.

[00:01:27:05] They're increasingly used in innovative ways, particularly within higher education. Our research sought to explore what can we learn from existing research in this area? What are the opportunities and challenges for portfolios in H.E. and secondary school, including for capturing and assessing transversal skills? What are the challenges of implementation and what recommendations can we make to facilitate successful implementation?

[00:01:54:23] We conducted a comprehensive review on the literature of portfolio theory and practice keywords were entered into the University of Cambridge online search platform, and this resulted in approximately ninety five articles being evaluated that were published between two thousand and twenty nineteen, written in English and conducted largely in Europe.

[00:02:15:18] We found that the majority of research was in the HE sector and there was a distinct paucity of research in secondary school settings. An important contribution of our review is the identification of several tensions in your portfolio theory and practice, which we believe have not been explicitly expressed elsewhere in this manner.

[00:02:35:15] The tensions were grouped into three broad themes the theory and research underpinning portfolios. And this theme is really about the fact that while they are strongly developed theoretical arguments behind the use of portfolios as a pedagogical tool and to measure transversal skills is unfortunately a lack of empirical research linking them to the intended outcomes, which constitutes a gap in the literature.

[00:02:58:18] The next theme is about the uses and purposes of portfolios, which consisted of several subthemes. The third theme was about the challenges and opportunities related to implementing e-portfolios, and this also consisted of several subthemes. The first group of tensions that I'd like to discuss are the uses and purposes of e-portfolios. The key tensions in this theme were true e-portfolios versus digital submission, and this includes discussion of the transversal skills of reflection and collaboration.

[00:03:30:14] Then there's control versus autonomous, individualised learning and summative versus formative assessment. And beyond this, we also found that portfolios can potentially support teaching and learning in various other ways beyond assessment. Are e-portfolio is simply a place to store work online, or are they something beyond that, something conceptually separate? Are we assessing and capturing the products of learning or the process, which is how the product was developed or how students understanding developed? How we conceptualise portfolios, influences, constructs that they're able to assess and capture.

[00:04:10:24] This table compares the differences between products and process portfolios, product e-portfolios also known as assessment e-portfolios, they consist of the finished product and therefore they have limited uses of the assessment of these final products. And these can be found in secondary schools process portfolios also known as learning portfolios or true e-portfolios, as well as consisting of final drafts. They also include drafts or unpolished work, which can capture the process of compiling the portfolio.

[00:04:41:23] Therefore, they can have broader uses. For example, looking at the process that produce artefacts, the story of the students learning journey. There were their reflections on their learning strategies or how they work developed over time. Some examples of this being used can be found in H.E.

[00:05:00:21] When turning to the transversal skill of reflection, it's very commonly included or assessed in your portfolios and there's a large body of literature about it. In essence, the artefacts uploaded or created in the e-portfolio can be reflected upon, and this reflection can be on the project themselves.

[00:05:17:28] For example, at the end of the process, reflecting on the strengths and weaknesses of the produce work, or it can be on the reflection on the process that produced the product. In this case, reflection could be done at multiple points in time during the learning process.

[00:05:35:05] This latter type of reflection on the process is arguably a deeper kind of reflection that promotes more continual engagement of students throughout the course. It can capture various concepts, such as the depth of their engagement, their self-awareness and their awareness of the learning process. And this can be argued to avoid the potential for a more complete assessment of student learning and personal growth and can encourage lifelong learning.

[00:06:01:15] This kind of e-portfolio was based over a period of time in which the student grew and developed during producing this work, and it supports reflection analogous to the metaphor of the learning journey.

[00:06:13:12] There are some challenges, however, with including reflection and assessment, such as constructing relevant variance and the potential benefits of washbag.

[00:06:22:18] Turning to collaboration. There was less literature about e-portfolios being used to assess collaboration, but that some theoretical arguments for how they could support it because portfolios can be shared or collaborative, meaning that multiple students can work together on a shared outcome. This can support aspects like negotiation, sharing resources and conflict resolution, and they can offer opportunities for both synchronous or asynchronous interaction.

[00:06:50:12] Another tension regarding the uses and purposes of e-portfolios is about control versus autonomy. How much control should the teacher or assessor exert over [00:07:00:00] the nature and contents of the e-portfolio? Or, in contrast, how much autonomy should students be given in the construction process? This can include questions of word or artefact limits, multimedia or software being constrained or specified, and whether the nature of the artefacts also constrained or specified in detail.

[00:07:19:28] There are benefits and drawbacks to each choice, and it depends on the intended purposes of the e-portfolio. If we have greater autonomy for students, this allows intrinsic motivation, individualised learning and greater engagement. But if we have greater control, this leads to greater uniformity of the assessment and more manageable and consistent marking.

[00:07:45:02] The next tension is between summative and formative assessment in e-portfolios, they can be used for both, but they will look different depending on what they used for when they're used for summative assessment. They are usually compiled at the end of a learning process. They have very prescribed [00:08:00:00] tasks with exemplars and detailed rubrics and mark schemes to score the outcomes, and students will carefully monitor what they contribute, whereas a formative e-portfolio would be maintained regularly throughout the process and would include less constrained tasks, possibly, and students might be more comfortable to share their true reflections. In both cases, ample technological support is required.

[00:08:23:14] Turning now to the implementation of e-portfolios, key tensions include conceptional versus technological change, teacher versus student ICT skills, engagement versus perceived burden, and pedagogy versus technology, conceptual versus technological change highlights that when we introduce change into educational settings, this requires understanding e-portfolios at a pedagogical level, but also requires having the right technological infrastructure to support it, which will vary with different context.

[00:08:54:10] There's also a potential gap between the skills of students and their teachers, which has implications for providing support and feedback. There are issues such as students with better technological abilities are being perceived to perform better in their e-portfolios. Student engagement is important for the success of e-portfolios and can influence their validity if students perceive it as an extra task to do rather than appreciating its value in the learning process.

[00:09:23:02] This could lead to superficial types of engagement, another tension between pedagogy and technology. And this reminds us that technology really expands our educational possibilities. But pedagogy should always come first. Once you have a strong pedagogical rationale for your e-portfolio, then you can select the right technology and implement it. This will lead to focused and coherent e-portfolios rather than odds and ends in a repository.

[00:09:52:25] This table, adapted from Lorenzo and its Wilson, shows different possible technological approaches and some examples of the advantages [00:10:00:00] and disadvantages of each, for example, home grown open source, commercial or common tools. And this table, adapted from several sources, provides different factors to take into consideration when choosing an approach to portfolio software.

[00:10:17:03] For example, you would need to consider the user experience, financial costs and the system access and security, amongst others. In conclusion, several tensions in e-portfolio theory and practice emerged under three themes, and these can guide decision making, design and implementation of e-portfolios. There was limited research, especially at school level. Most of the research was on HE and it was often methodologically limited in the sense that it was based on case studies and perception studies.

[00:10:46:28] However, the limited literature does indicate that e-portfolios have the potential for capturing and assessing transversal skills, definitely assessing reflection and to a lesser extent, collaboration. E-portfolios [00:11:00:00] provide an opportunity to make the role of reflection and learning explicit, and they can support collaboration by creating opportunities for students to share, discuss and work together on outcomes. Process oriented e-portfolios have the potential to capture constructs akin to the students learning journey.

[00:11:20:27] Key conclusions in terms of implementation is that each e-portfolio should be embedded in the design phase and take note of the contextual factors which will influence [00:11:30:00] the extent of change and support required. Guidelines were given for selecting an approach to portfolio software and high quality technological support is required. Lastly, student engagement is crucial for successful portfolio implementation.

[00:11:46:04] And that brings me to the end of my presentation. Thank you very much for listening and I welcome any comments and discussion points. Please contact me if you would like to discuss this work further. Thank you.

Return to top

Research Matters

Research Matters 32 promo image

Research Matters is our free biannual publication which allows us to share our assessment research, in a range of fields, with the wider assessment community.