I shall start this evening by saying a few words about Cambridge Assessment. Cambridge Assessment is a Department of the University of Cambridge and a not-for-profit organisation. Each year, our three exam board subsidiaries, OCR, Cambridge ESOL and Cambridge International Exams conduct some 20 million exams in 150 countries around the world, generating revenues of approximately £200m. Our exams cover more or less every type of assessment, and operating globally in this way we work directly with Ministries, with state and independent school systems, and in both regulated and non-regulated environments. The question of intelligent regulation and of how to manage risk and secure trust is, therefore, one that interests us vitally, and I am delighted to have been extended the opportunity by QCA to give an Awarding Body perspective as, although it may seem a dry subject, what is at stake is nothing less than the quality of our children’s education.
I should say next that I warmly welcome the recent announcement of the plan to re-organise QCA and to make regulation of tests and qualifications the responsibility of a separate and independent Regulator in order to put beyond question the independence of the public guardian of standards. To recap, the new Regulator will be responsible for regulating the awarding body market to ensure that it develops in a way that promotes innovation and increases efficiency, and for ensuring good value for money from public investment in qualifications. The new Regulator is expected to report regularly to Parliament and a key goal of the reform is to create long-term trust and credibility through the development of transparent frameworks for decision-making along the lines of the Food Standards Agency and Bank of England’s Monetary Policy Committee.
The new Regulator will, therefore, be both an economic and a quality regulator while QCA will retain its responsibility for curriculum development and advice, National Curriculum test development and delivery, the delivery of reforms to vocational education and, it is proposed, the development of criteria for public qualifications.
Will this work? Will these new arrangements achieve the objective of creating long term trust and stability?
Qualifications, of course, reflect the wider social and policy environment, and this is one where major change is an almost constant feature, not least because qualifications represent one of the more easily manipulable supply side levers in an education where reform is so complex. Inasmuch therefore as this is the first major change to the constitution of QCA since it was set up in 1997, this makes it almost a beacon of stability. In the summer period, the Education Department has changed names three times (from DfEE to DfES to DCSF) and in the five and a half years since I started doing this job there have been five Secretaries of State. In the field of qualifications with which we are concerned here there has been the introduction of C2K, major revisions to GCSE, the change of GNVQ to AVCE, the change of Core Skills to Key Skills to Functional Skills, changes in A level to introduce “stretch and challenge” and to reduce A level units from six to four, and the development and launch of Diplomas, originally planned in thirteen occupationally linked areas, now also in humanities, language and science. This has happened side-by-side with a major investment in technology which has seen examinations move away from being, in Ken’s phrase (with which I don’t wholly concur) a “cottage industry” to one where electronic marking is more or less general practice and where a number of high stakes assessments are now being offered on line.
What does this mean for the Regulator? I believe part of the new Regulator’s role must be to emphasise the need for stability and if there is to be change, the need for it to be carefully managed, not least because this is the best way to protect learners’ interests. The current set up, with QCA being responsible at the same time for delivering change within the qualifications system and for maintaining standards and answering for both of these things to Ministers, has permitted a model to develop where qualifications reform is usually disruptive rather than incremental. The proposed new arrangements, with the Regulator reporting to Parliament and QCA continuing to work closely with DCSF and DIUS, mean that the new Regulator can undertake the job of managing political expectations about what rate of change is prudent and achievable and of ensuring that there is proper political understanding of the consequences of change and the risks associated with it.
This is important because continual change erodes confidence and creates confusion. System change is the point of maximum risk in the delivery of large scale public examinations, and this was well illustrated by the problems in Summer 2002, the first full year of the Curriculum 2000 changes, which had much to do with a lack of understanding of some of the implications for standards of the new modular structure. There is also the issue of currency. A former Secretary to the Oxford Delegacy of Local Examinations wrote last week that “Bitter experience showed my colleagues and me that it takes some 25 years for new examinations to be fully accepted by the academic and business communities”, while the Times carried a headline article titled “Don’t abandon A levels, they deliver world-class students, says Yale chief”, who noted that the depth of knowledge displayed by the best A level students made them prime targets for America’s Ivy League Universities and that “it would be a shame to lose that rigour”. These comments both emphasise the need to safeguard carefully the value and currency that resides in existing qualifications and to be cautious about change. This is not an argument I stress for standing still but it is an argument for making change only once it is fully thought through and there is a clear understanding of the benefits, drawbacks and risks of such change. I believe it must be a central part of the new Regulator’s approach to intelligent regulation to create the conditions for such understanding and to feel able publicly to advise Ministers where system stability is being jeopardised as a result of too much change, or as a result of change being put through too quickly.
There is also a specific element of the Secretary of State’s proposals that is relevant in this context. It is at present contemplated that QCA should retain responsibility for developing the criteria for public qualifications, such as GCSEs, A-levels and 14-19 Diplomas. However, I think we are more likely to achieve the necessary balance - between curriculum change and fit for purpose assessment leading to a stable well regarded qualifications regime which enjoys public confidence - if responsibility for assessment design is left with those who are accountable for the integrity and stability of the system as a whole rather than those who are responsible for delivering curriculum change. The most obvious example of the problems that arise when these functions are not separate is again Curriculum 2000. Despite repeated and extensive advice to the effect that it would lead to difficulty in maintaining the standard, the decision was taken, largely for expedient reasons, that AS and A2 units should have equal 50% weightings when it came to determining the final grade. The consequence was a significant, one off hike in attainment levels that was the backdrop to the A level crisis of 2002 and arguably led to a sustained crisis of confidence in A levels, some of the consequences of which we continue to deal with today. A more recent example is some of the work that went into “stretch and challenge” – the changes to A level designed to make them more academically demanding at the top end and to make sure the result scheme would provide additional differentiation by introducing an A* grade. The current set up of QCA and the responsibility Ministers have for such matters meant that there was a lack of transparency and timeliness in the decision making process. My point is that it is absolutely proper and legitimate that there should be a political decision that there needs to be more “Stretch and challenge”, but once that decision has been taken, it should then be for the new Regulator to work with Awarding Bodies to determine how best to implement it without any further political involvement. The current proposal that QCA rather than the new Regulator should have responsibility for developing the criteria for public qualifications does not seem to me to achieve this or guarantee the necessary degree of ministerial distance since. If the system is set up so that it removes the Regulator from the qualifications development process until the very end it risks depriving that Regulator of its primary means of ensuring that reliable, high standard and fit for purpose qualifications are developed. We shall therefore be pressing for this to be changed when DCSF launches its consultation later this month and for the responsibility for developing the criteria for public qualifications to rest with the new Regulator.
I have talked about the Regulator’s role in managing change and managing the political drive for change so that it does not destabilise the system. I have also commented on the need to ensure that the development of the criteria for public qualifications should reside with the Regulator rather than with QCA. I now want to come on to the contentious question of standards.
The announcement of the new arrangements for an independent Regulator makes it clear that improving confidence in standards is one of the principal reasons for the changes. This is timely. Notwithstanding Professor Levin of Yale’s comments quoted above, there is no doubt that confidence in A levels particularly and also GCSEs has suffered over recent years. A 2005 ICM poll for Reform, for example, found that nearly half the public think A levels have become easier over the last ten to fifteen years. We are all, also, familiar with some of the negative press coverage that greets the results every summer. It is difficult to be precise about exactly what has happened and the whole issue of standards over time is notoriously complex and technically challenging. However, if one takes subjects like maths or science where direct comparisons can be easier to make it is hard not to be troubled for example by some of the work done by the CEM Centre at Durham using standard test items over a number of years and correlating these to A level results or by comments such as those of Professor Sir Peter Williams, Chancellor of Leicester University and Chair of the Advisory Committee on Mathematics and Education who this Summer said “Over twenty or thirty years, I don’t think there is any doubt whatsoever that absolute A-level standards have fallen. They have edged south, continuously over a long period of time. I think all university academics and a good proportion of sixth-form teachers would agree with my assertion.”
My next slide, which goes back to 1998, shows some of the Ministerial responses to this criticism. It is not part of my purpose this evening to say which of these accounts I think is correct. It is possible, however, to see that there is a real need for a better informed and more dispassionate debate about the whole issue, and that we cannot, in the face of such a divergence of views, simply take refuge in the technical difficulty of the issues involved. I believe we all - QCA, the Awarding Bodies, politicians and DCSF in its various guises - have been remiss in not being readier to debate the impact of changes in A level, perhaps not least because we may not have felt a need to do so operating within a culture where there has been an expectation of consistently improving levels of attainment. Inevitably, however, given how long it has been around, the nature of A levels has changed. I am thinking specifically of things like the change from its being an exam designed to select applicants for admission to University to a General Certificate of Education for Year 13 leavers, of the impact of higher post 16 staying on rates, of the consequence of increased rates of participation in Higher Education. I think there have also been societal changes that have affected attitudes to qualifications. One example is the drive for increased equity which is arguably one of the factors behind the development of highly detailed mark schemes and specific assessment criteria as people are no longer ready to submit themselves to more flexible but less well defined assessment regimes that use sampling and are thus less predictable and more judgement based. This is partly, of course, because we live in a society that is generally less comfortable with judgement, certainly less comfortable with, as it were, black box professional judgement than was the case a few years ago, and to which the concept of selection, after all one of the functions for which A level was originally designed, is uncongenial. It is also happening at a time when the reality of global mobility and a rapidly changing economy means that more and more people are taking more and more qualifications, and that these are becoming more and more high stakes.
These are all serious and complex issues and I don’t believe it is possible to understand properly what has happened to A levels without taking such factors into account or to protect their currency without discussing such matters and ensuring a good public understanding of them. Although QCA has done good work on individual comparability studies and subject reviews, this has not provided a platform for considering the totality of the evidence, not least because of the changes in different specifications that take place all the time. I think it is critical that the new Regulator has as a central part of its role a formal responsibility to ensure a well informed public understanding of standards. This needs to be supported by funding for a significant and permanent research capability and the creation of a substantial national script archive. It is an essential requirement if trust in standards is to be restored.
This is all the more important because we are about to embark on a period of such major change in the UK’s qualifications regime. We are moving from a relatively straightforward environment in 14-19 where Key Stage 3 is followed by GCSE and A level and a significant element of the cohort leaves at 16, to one where the leaving age is effectively to be raised to 18 and where there will be GCSEs, A levels, IGCSEs, Diplomas, a more widely available International Baccalaureate, and the Cambridge Pre U. I welcome the choice and pluralism that is represented by these arrangements and believe it is a positive educational development. However, it also introduces significant scope for confusion as to how the qualifications compare and what standards should be used as metrics for measuring attainment. It will be a major part of the new Regulator’s task to manage this, and to ensure that there is enough understanding of the new qualifications and the way in which they relate to each other that they come rapidly to enjoy public, employer and HE confidence, an end, incidentally, which might also be served by the new Regulator taking on responsibility for League Tables.
I have so far focused on the quality elements of the Regulator’s role and I want now to turn to the question of economic regulation. The Secretary of State’s announcement makes it clear that an important part of the new Regulator’s role will be to ensure that public investment in qualifications provides good value for money. In contrast this barely featured in the 1997 Education Act which established QCA, there being only one mention, a provision that QCA could, as part of its accreditation, impose a limit on the fee that could be charged for any accredited award or qualification, a power as far as I am aware which has never been used.
The figures for expenditure on exams are nonetheless significant and have been growing. Much of this growth is a function I believe of the increase in the number and complexity of exams being taken, at least partly in response to an increasingly powerful government emphasis on gaining qualifications. A level entries, for example, have risen from just under 180,000 18-year old candidates in 1996 to just over 220,000 in 2006 and GCSE from 500,000 16-year old candidates in 1994 to 615,000 in 2006. As a result QCA has, during the period, begun to interest itself both in the structure and efficiency of the Awarding Body market and more recently has been remitted by the Government to look into reasonableness of fees charged for A levels and GCSEs. In 2004, it published a report it had commissioned by PWC which estimated the total cost of the regulated school exam system, including the NCT tests carried out by QCA’s NAA subsidiary, to be around £610m (though this included a significant amount of estimated indirect cost) . More recently, in 2006, the TES published an analysis of secondary school spending which showed secondary schools spending £197 million on exam fees in 2004-5. Interestingly this was more than was spent on books, where expenditure for primary and secondary schools was estimated at around £150m, but somewhat less than ICT, estimated at £426m, insurance at £311m and school food, shown at £491m though I have seen estimates elsewhere at almost double this.
The Awarding Body market is nevertheless highly unusual. Although there are well over 100 accredited awarding bodies offering mainly vocational qualifications, and many more offering non-accredited qualifications, there are only five awarding bodies for GCSEs and A levels (three in England, and one each in Wales and Northern Ireland), of which all but one are not-for-profit organisations. The QCA has characterised this as an “oligopoly” (though by way of comparison the markets for school meals are dominated by just two companies and for school books by just three) and it this year commissioned a report by PKF to look into the reasonableness of fees. This concluded that “At the highest level the fees charged by Awarding Bodies are considered to be reasonable as there are no significant profits being made. Minimal profitability is made by the Awarding Bodies on GCSEs and A levels, with GCSEs being more profitable than A levels. A levels are generally loss making. For both GCSEs and A levels there are only a few profitable subjects that subsidise the rest of the loss making subjects”. This does not sound to me like the economics of conventional oligopoly and indeed, given the risks involved, there might be an argument that the returns are disproportionately low.
What does this mean for economic regulation? I think it is entirely proper, given the considerable sums of public money involved, that the subject receives attention. However, it is important that this is considered in context and that some of the other factors driving up qualification costs are taken into account. These include first of all numbers. As I mentioned above, the more fluid economy, changing career patterns, global mobility and higher school participation rates all mean that learners are taking more qualifications. This is an international trend, and it is further supported by the philosophy of unitisation and creditisation behind programmes such as the Framework for Achievement and reforms to the National Qualifications Framework. The suggestion that we can dramatically cut the volume of assessment by having more teacher based assessment or by creating a new class of Chartered Assessors fails, I believe, to reflect these realities. Qualifications are also becoming more complicated, the new Diplomas being a case in point, where the complex nature of the award, the involvement of a variety of different organisations as component awarding bodies and the use of different assessment types will all inevitably put upward pressure on costs. In addition, more resits are being taken - appropriate, perhaps, from an equity point of view, but expensive, and the system is becoming more complex as subject choices and combinations increase, again a good thing from an educational point of view, but a factor driving up administration costs.
Another major factor is technology. This is expensive. QCA itself spent £23m on the Key Stage 3 ICT pilot, now to be used for formative assessment, and the English Awarding Bodies have invested tens of millions of pounds in the development of electronic script marking. A further round of investment is now also underway to develop the capacity for computer based testing, and in the Cambridge Assessment Group both OCR and Cambridge International exams have now offered GCSEs in an on-line format. Based on my international experience I would say that the English system now has some of the most sophisticated technology in the world and this has been developed as a natural result of competition between the Awarding Bodies and with relatively little regulatory prompting or intervention. Indeed, I would suggest that it was Pearson’s acquisition of Edexcel rather than Ken’s admonitions to a “cottage industry” that has had the most powerful effect in terms of driving modernisation.
The Secretary of State’s announcement recognises this change in the Awarding Body market and notes that “we need to ensure that it develops in a way that promotes innovation, reduces burden and increases efficiency”. My plea would be that this is achieved through a relatively light touch approach to regulation, which has a clear vision of what it is that it is setting out to achieve. It will need to be supported by a suitable regulatory doctrine that takes account of some of the peculiar characteristics of the market. It is after all unusual, possibly unique, for a regulator to be regulating a market where not for profits compete with commercial concerns and where, as PKF identified above, a number of services are knowingly supplied at a loss. It is also important to recognise that the English Awarding Body system represents a rich public resource for the development of new qualifications. This is not always remembered or appreciated, though, it is historically true that many of the innovations we now take for granted such as modular A levels or coursework originated with Awarding Bodies, as did such influential curricular programmes as SMP maths and Nuffield and Salters Science. There is a real danger that an ill thought through pricing regime, or the inappropriate wholesale transfer of a regulatory model used in another sector such as utilities, could end up monetising the goodwill in the system by requiring that everything be accounted for. This would make many of the existing subsidies visible and explicit but would ultimately disrupt the ecology of a system that at present does a fine job supporting minority subjects and devises assessments based on educational rather than economic values.
I think it also follows from this that while the Regulator has a role in managing the market it is not part of its responsibility to seek to restructure it except in the event of catastrophic market failure. The current structure – oligopoly if you wish – arose out of a government prompted re-structuring in the 90s which saw the number of awarding bodies for general qualifications in England reduce from 12 in 1993 to the present number of 3 and it is interesting in this context that recent procurement in relation to National Curriculum tests has in contrast actually sought to increase rather than reduce the number of players. The same is also true of the number of qualifications. The goal of “coherence”, alluded to only very briefly in an aspirational sort of way in the 1997 Act where it is stated that the QCA “shall exercise its functions with a view to promoting quality and coherence” has since become a significant programme, informing much of the work behind the Framework for Achievement and changes to the Qualifications and Credit Framework. Poorly based estimates of the number of qualifications abound, some as high as 20000. However, as of May 2007 the National Database of Accredited Qualifications showed only around 4800 accredited vocational qualifications and some 938 mainstream general academic qualifications, a very comparable figure for example to Germany, often held out as a model of how such things should be organised, where there are in fact over 4000 registered vocational qualifications. Indeed PWC carried out an employer survey in connection with an April 2005 report it prepared for QCA on “The Market for Qualifications in the UK” in which more than 90% of respondents stated that current qualifications met their business needs. The requirement for the Regulator to “reduce(s) burden and increase efficiency” should not therefore be taken as a justification for major supply side intervention in the market, either to impose coherence or to control the number of market participants.
Finally, I should like to say a few words about governance as there is clearly now an opportunity to review existing arrangements. Particular areas where there is scope for improvement are first of all a much greater and more rigorous use of Regulatory Impact Assessments. New initiatives have not necessarily been subject to these in the past and this has sometimes meant that there has not been a proper understanding of the financial, operational and other implications that they have had, some of the initiatives over the last two or three years on bureaucracy being particular examples. This is also important in the now not uncommon situation where the precise effect of new compliance obligations, for example in relation to the DDA, is unlikely to be clear until some considerable length of time and several test cases after the passage of the legislation.
I hope also that the line of accountability to Parliament will lead to a reduction in the number of remit letters. These have acted as an engine of uncertainty and regulatory creep in the past and should not be needed if overall aims and objectives have been properly agreed and clearly stated in advance.
In summary therefore, the changes planned for the regulatory regime provide an opportunity to remedy weaknesses inherent in the present set up and to make some of the changes that are needed to restore trust and confidence. Specifically, it is essential that the Regulator take a more proactive stance in managing political stakeholders and the rate of change – there is a primary responsibility to safeguard the value of qualifications currency, and this has priority over the execution of change in the event these conflict. It is also fundamental, and perhaps the most important single element of the Regulator’s quality remit, that it engage actively and wholeheartedly in the debate about standards. Uncertainty about this is corrosive and there needs to be an informed and active public discourse, not least about some of the trade offs between precision in standards and a system that is flexible and will accommodate change. On the economic front the brief is different. Here the need is to develop a new light touch regulatory doctrine that recognises the unique features of the English Awarding Body system and promotes modernisation and efficiency without either discouraging investment or penalising diversity through the pursuit of administrative neatness. It is a considerable challenge and one where both Cambridge and I am sure the other Awarding Bodies look forward to working with both QCA’s successor body and the new Regulator. Thank you.
Simon Lebus
Group Chief Executive
Cambridge Assessment
8 November 2007