WEBVTT

00:00:00.000 --> 00:00:14.500
International surveys are gaining in popularity now in respect of producing information which

00:00:14.500 --> 00:00:19.920
individual nations can reflect on in terms of their own performance and then think about how

00:00:19.920 --> 00:00:27.320
say complex factors mixed together in different systems to give them either poor or high performance.

00:00:27.320 --> 00:00:34.480
I think these surveys are a good tool for pointing out to us those systems that are improving

00:00:34.468 --> 00:00:38.348
and those systems that have put in place policy which have led to improvements.

00:00:38.348 --> 00:00:42.068
And that in itself is a very, very important mechanism.

00:00:43.188 --> 00:00:46.188
Obviously each country has its own culture, its own history,

00:00:46.188 --> 00:00:49.788
it has its own complex conjunction of factors

00:00:49.788 --> 00:00:53.308
which give rise to performance in their setting.

00:00:53.308 --> 00:00:56.588
Now the good thing is I've been able to say that.

00:00:56.588 --> 00:00:59.628
You know, it's a recognition that that's happening.

00:00:59.628 --> 00:01:02.908
And that can help with the sensitivity with which we approach

00:01:02.908 --> 00:01:06.888
the outcomes which these surveys produce.

00:01:06.888 --> 00:01:08.928
They are relatively new,

00:01:08.935 --> 00:01:15.775
So governments are not necessarily tuned to using the outcomes with precision and fidelity.

00:01:15.775 --> 00:01:21.695
And David Rafe has, I think, introduced a very elegant phrase, which is policy learning.

00:01:21.695 --> 00:01:26.555
In other words, you look at these other systems, you look at the way in which things interact,

00:01:26.555 --> 00:01:30.355
and then you can reflect on your own system and what interventions might push your own

00:01:30.355 --> 00:01:33.275
system in the right direction.

00:01:33.275 --> 00:01:37.655
These surveys have to select things that are in common across systems.

00:01:37.655 --> 00:01:43.415
kind of a prerequisite that the items make sense across the totality of those participants.

00:01:43.403 --> 00:01:47.403
in the sample, so they do tend to converge on certain things.

00:01:47.403 --> 00:01:52.403
We do have to watch this problem of countries increasingly converging

00:01:52.403 --> 00:01:55.403
just on the items within the surveys.

00:01:55.403 --> 00:01:59.403
That would be undue narrowing of the curriculum focus of nations,

00:01:59.403 --> 00:02:04.403
and it's something which OECD and researchers are actually warning against.

00:02:04.403 --> 00:02:09.403
The underlying methodology and the assumptions which drive these surveys

00:02:09.403 --> 00:02:13.403
should of course be subject to constant scientific scrutiny.

00:02:13.403 --> 00:02:17.863
The improvement in the methodology is actually a double-edged sword.

00:02:17.871 --> 00:02:21.471
because as the surveys get better and they're more finely tuned,

00:02:22.151 --> 00:02:25.011
of course you're not actually then comparing light with light

00:02:25.011 --> 00:02:26.791
because you've changed the measurement instruments.

00:02:27.471 --> 00:02:30.911
So this methodology area is one which is fraught with problems.

00:02:31.471 --> 00:02:34.531
But I'm convinced that we're becoming more and more elegant

00:02:34.531 --> 00:02:37.191
and more and more refined in the way in which we're running these surveys.

00:02:37.731 --> 00:02:41.331
It's all looking pretty good in terms of the way in which

00:02:41.331 --> 00:02:43.351
we're building up a science around these surveys.

00:02:47.871 --> 00:02:55.131
by