[00:00:07:20] Hello, I'm Emma from Cambridge Assessment, and today I'm
going to be talking to you about the results of some research that I
conducted with my colleague Stuart Shaw about using portfolios to capture
and assess transversal skills, tensions in theory and practice.
[00:00:23:27] Many students today live in a multimedia hypertextual and
digitalised world, but often the assessment and learning
methods they experience can be somewhat of an anachronism that can be out
of step with their technological lives. Therefore, proponents of portfolios
call for the use of technological methods to transform teaching, learning
and assessment.
[00:00:44:19] Beyond this, assessment methods are more geared toward
assessing products rather than processes like reflection and collaboration,
and e-portfolios have been put forward as potential tools to assist these
processes.
[00:00:57:13] These are seen to be important skills in today's knowledge
[00:01:00:00] economy, and the value of promoting lifelong learning is more
important than ever.
[00:01:05:10] However, there are major barriers, including the fact that
not all students have the same access to technology.
[00:01:13:04] Several different definitions of portfolios exist, but they
generally refer to purposeful collections of information and digital
artefacts that demonstrate developmental evidence, learning outcomes,
skills or competencies.
[00:01:27:05] They're increasingly used in innovative ways, particularly within higher education. Our research sought to explore what
can we learn from existing research in this area? What are the
opportunities and challenges for portfolios in H.E. and secondary school,
including for capturing and assessing transversal skills? What are the
challenges of implementation and what recommendations can we make to
facilitate successful implementation?
[00:01:54:23] We conducted a comprehensive review on the literature of
portfolio theory and practice keywords were entered into the
University of Cambridge online search platform, and this resulted in
approximately ninety five articles being evaluated that were published
between two thousand and twenty nineteen, written in English and conducted
largely in Europe.
[00:02:15:18] We found that the majority of research was in the HE sector
and there was a distinct paucity of research in secondary school settings.
An important contribution of our review is the identification of several
tensions in your portfolio theory and practice, which we
believe have not been explicitly expressed elsewhere in this manner.
[00:02:35:15] The tensions were grouped into three broad themes the theory
and research underpinning portfolios. And this theme is really about the
fact that while they are strongly developed theoretical arguments behind
the use of portfolios as a pedagogical tool and to measure transversal
skills is unfortunately a lack of empirical research linking them to the
intended outcomes, which constitutes a gap in the literature.
[00:02:58:18] The next theme is about the uses and purposes
of portfolios, which consisted of several subthemes. The third theme was
about the challenges and opportunities related to implementing
e-portfolios, and this also consisted of several subthemes. The first group
of tensions that I'd like to discuss are the uses and purposes of
e-portfolios. The key tensions in this theme were true e-portfolios versus
digital submission, and this includes discussion of the transversal skills
of reflection and collaboration.
[00:03:30:14] Then there's control versus autonomous,
individualised learning and summative versus formative assessment. And
beyond this, we also found that portfolios can potentially support teaching
and learning in various other ways beyond assessment. Are e-portfolio is
simply a place to store work online, or are they something beyond that,
something conceptually separate? Are we assessing and capturing the
products of learning or the process, which is how the product was developed or how students understanding developed? How we conceptualise
portfolios, influences, constructs that they're able to assess and capture.
[00:04:10:24] This table compares the differences between products and
process portfolios, product e-portfolios also known as assessment
e-portfolios, they consist of the finished product and therefore they have
limited uses of the assessment of these final products. And these can be
found in secondary schools process portfolios also known as
learning portfolios or true e-portfolios, as well as consisting of final
drafts. They also include drafts or unpolished work, which can capture the
process of compiling the portfolio.
[00:04:41:23] Therefore, they can have broader uses. For example, looking
at the process that produce artefacts, the story of the students learning
journey. There were their reflections on their learning strategies or how
they work developed over time. Some examples of this being used can be
found in H.E.
[00:05:00:21] When turning to the transversal skill of
reflection, it's very commonly included or assessed in your portfolios and
there's a large body of literature about it. In essence, the artefacts
uploaded or created in the e-portfolio can be reflected upon, and this
reflection can be on the project themselves.
[00:05:17:28] For example, at the end of the process, reflecting on the
strengths and weaknesses of the produce work, or it can be on the
reflection on the process that produced the product. In this case,
reflection could be done at multiple points in time during
the learning process.
[00:05:35:05] This latter type of reflection on the process is arguably a
deeper kind of reflection that promotes more continual engagement of
students throughout the course. It can capture various concepts, such as
the depth of their engagement, their self-awareness and their awareness of
the learning process. And this can be argued to avoid the potential for a
more complete assessment of student learning and personal growth and can
encourage lifelong learning.
[00:06:01:15] This kind of e-portfolio was based over a period of time in
which the student grew and developed during producing this work, and it
supports reflection analogous to the metaphor of the learning journey.
[00:06:13:12] There are some challenges, however, with including reflection
and assessment, such as constructing relevant variance and the potential
benefits of washbag.
[00:06:22:18] Turning to collaboration. There was less literature about
e-portfolios being used to assess collaboration, but that some theoretical arguments for how they could support it because portfolios
can be shared or collaborative, meaning that multiple students can work
together on a shared outcome. This can support aspects like negotiation,
sharing resources and conflict resolution, and they can offer opportunities
for both synchronous or asynchronous interaction.
[00:06:50:12] Another tension regarding the uses and purposes of
e-portfolios is about control versus autonomy. How much control should the
teacher or assessor exert over [00:07:00:00] the nature and contents of the
e-portfolio? Or, in contrast, how much autonomy should students be given in
the construction process? This can include questions of word or artefact
limits, multimedia or software being constrained or specified, and whether
the nature of the artefacts also constrained or specified in detail.
[00:07:19:28] There are benefits and drawbacks to each choice, and it
depends on the intended purposes of the e-portfolio. If we have greater
autonomy for students, this allows intrinsic motivation, individualised learning and greater engagement. But if we have greater
control, this leads to greater uniformity of the assessment and more
manageable and consistent marking.
[00:07:45:02] The next tension is between summative and formative
assessment in e-portfolios, they can be used for both, but they will look
different depending on what they used for when they're used for summative
assessment. They are usually compiled at the end of a learning process.
They have very prescribed [00:08:00:00] tasks with exemplars and detailed
rubrics and mark schemes to score the outcomes, and students will carefully
monitor what they contribute, whereas a formative e-portfolio would be
maintained regularly throughout the process and would include less
constrained tasks, possibly, and students might be more comfortable to
share their true reflections. In both cases, ample technological support is
required.
[00:08:23:14] Turning now to the implementation of e-portfolios, key
tensions include conceptional versus technological change, teacher versus student ICT skills, engagement versus perceived burden, and
pedagogy versus technology, conceptual versus technological change
highlights that when we introduce change into educational settings, this
requires understanding e-portfolios at a pedagogical level, but also
requires having the right technological infrastructure to support it, which
will vary with different context.
[00:08:54:10] There's also a potential gap between the skills of students
and their teachers, which has implications for providing support and feedback. There are issues such as students with better
technological abilities are being perceived to perform better in their
e-portfolios. Student engagement is important for the success of
e-portfolios and can influence their validity if students perceive it as an
extra task to do rather than appreciating its value in the learning
process.
[00:09:23:02] This could lead to superficial types of engagement, another
tension between pedagogy and technology. And this reminds us
that technology really expands our educational possibilities. But pedagogy
should always come first. Once you have a strong pedagogical rationale for
your e-portfolio, then you can select the right technology and implement
it. This will lead to focused and coherent e-portfolios rather than odds
and ends in a repository.
[00:09:52:25] This table, adapted from Lorenzo and its Wilson, shows
different possible technological approaches and some examples of the
advantages [00:10:00:00] and disadvantages of each, for example, home grown
open source, commercial or common tools. And this table, adapted from
several sources, provides different factors to take into consideration when
choosing an approach to portfolio software.
[00:10:17:03] For example, you would need to consider the user experience,
financial costs and the system access and security, amongst others. In
conclusion, several tensions in e-portfolio theory and practice emerged under three themes, and these can guide decision making,
design and implementation of e-portfolios. There was limited research,
especially at school level. Most of the research was on HE and it was often
methodologically limited in the sense that it was based on case studies and
perception studies.
[00:10:46:28] However, the limited literature does indicate that
e-portfolios have the potential for capturing and assessing transversal
skills, definitely assessing reflection and to a lesser extent,
collaboration. E-portfolios [00:11:00:00] provide an opportunity to make
the role of reflection and learning explicit, and they can support
collaboration by creating opportunities for students to share, discuss and
work together on outcomes. Process oriented e-portfolios have the potential
to capture constructs akin to the students learning journey.
[00:11:20:27] Key conclusions in terms of implementation is that each
e-portfolio should be embedded in the design phase and take note of the
contextual factors which will influence [00:11:30:00] the extent of change
and support required. Guidelines were given for selecting an approach to
portfolio software and high quality technological support is required.
Lastly, student engagement is crucial for successful portfolio
implementation.
[00:11:46:04] And that brings me to the end of my presentation. Thank you
very much for listening and I welcome any comments and discussion points.
Please contact me if you would like to discuss this work further. Thank
you.
Return to top