1
00:00:10,419 --> 00:00:15,880
Thank you. Let's talk a bit more about the PISA assessment.

2
00:00:16,879 --> 00:00:20,100
I think Minister van Grieken made an important point.

3
00:00:20,219 --> 00:00:24,620
We have not been able to assess everything that is important in life.

4
00:00:24,820 --> 00:00:30,980
We have started, but some things that we do know are very, very important for the success of people.

5
00:00:30,980 --> 00:00:41,780
And that's the capacity of students to access, manage, integrate, evaluate, reflect on written information, literacy.

6
00:00:43,299 --> 00:00:54,600
It's the capacity of people to think quantitatively, to reason analytically, to find ways to model the world in mathematic terms.

7
00:00:54,700 --> 00:00:55,340
That's mathematics.

8
00:00:55,340 --> 00:01:00,119
and it's the capacity of people to think like a scientist

9
00:01:00,119 --> 00:01:03,840
in a world that is increasingly dominated by scientific aspects.

10
00:01:04,939 --> 00:01:06,500
That's not everything, but it's a lot.

11
00:01:07,040 --> 00:01:11,079
Actually, we have data that show that those kinds of skills,

12
00:01:11,420 --> 00:01:16,780
knowledge and skills, are highly predictive for people's later lives.

13
00:01:17,299 --> 00:01:20,079
In fact, you can see this is true for the economic life

14
00:01:20,079 --> 00:01:22,799
when you look at the earnings of people, employment of people,

15
00:01:22,900 --> 00:01:24,120
very strong links to this.

16
00:01:24,120 --> 00:01:27,260
but it's also true for many social aspects of life.

17
00:01:28,400 --> 00:01:30,980
You may be surprised, but we find a strong link

18
00:01:30,980 --> 00:01:33,760
between the skills that people have in mathematics

19
00:01:33,760 --> 00:01:37,519
and the extent to which they participate in society.

20
00:01:38,480 --> 00:01:40,680
People at the high end of the skill distribution

21
00:01:40,680 --> 00:01:43,319
see themselves often as actors in social processes.

22
00:01:43,500 --> 00:01:44,640
They believe they can do things.

23
00:01:45,760 --> 00:01:48,159
People at the low end of the skill distribution

24
00:01:48,159 --> 00:01:51,859
see themselves often as objects of political process.

25
00:01:51,859 --> 00:01:55,620
They do not feel that sense of empowerment.

26
00:01:55,620 --> 00:02:01,280
Those foundation skills are important ingredients for people's active participation in our life

27
00:02:01,280 --> 00:02:03,900
and societies.

28
00:02:03,900 --> 00:02:07,859
That being said, PISA is moving forward also.

29
00:02:07,859 --> 00:02:13,039
This year, a little bit later, we're going to launch the first results from our PISA

30
00:02:13,039 --> 00:02:15,099
assessment of social skills.

31
00:02:15,099 --> 00:02:24,439
In 2015, we assessed not only whether students can solve problems individually, that's very important,

32
00:02:25,259 --> 00:02:30,360
but also whether they can effectively collaborate with others to share their knowledge, integrate their knowledge.

33
00:02:30,439 --> 00:02:34,800
Because today, the world no longer rewards people just what they do individually,

34
00:02:34,800 --> 00:02:40,300
but very much so to what extent they can actually collaborate, compete, connect with people,

35
00:02:40,819 --> 00:02:42,479
connect with people who think differently.

36
00:02:42,479 --> 00:02:48,120
So again, as you can see, the world is changing, demanding a greater amount of social skills.

37
00:02:48,340 --> 00:02:53,460
PISA is changing, trying to reflect those things in the way we assess student knowledge and skills.

38
00:02:53,620 --> 00:03:01,699
And you will find that kind of collaborative problem-solving skill test also in future PISA for Schools tests.

39
00:03:02,960 --> 00:03:11,159
We're even moving one step further in the 2018 assessment towards looking at global competency.

40
00:03:11,159 --> 00:03:17,159
What do I mean by this? It's not about people who speak multiple languages or who travel the world.

41
00:03:17,159 --> 00:03:23,159
It's about the capacity of individuals to look at the world through different perspectives,

42
00:03:23,159 --> 00:03:28,159
through different lenses, through different ways of thinking, different ways of walking,

43
00:03:28,159 --> 00:03:32,159
to appreciate different ideas, different values, different cultures of people.

44
00:03:32,159 --> 00:03:38,159
And that is something that is, you know, in a world that is becoming increasingly diverse,

45
00:03:38,159 --> 00:03:44,159
in important skillset. So we want to reflect that in the PISA assessment.

46
00:03:44,159 --> 00:03:49,159
We've also put emphasis on looking at some of the social and emotional skills

47
00:03:49,159 --> 00:03:55,159
that make people successful. Effort, persistence. I'm going to actually show you some results

48
00:03:55,159 --> 00:04:00,159
from our current PISA test that actually show you sometimes, you know, cognitive skills

49
00:04:00,159 --> 00:04:05,159
and social and emotional skills often go together, but not always.

50
00:04:05,159 --> 00:04:10,379
always. And we need to look at those kinds of differences. There's a lot of effort being

51
00:04:10,379 --> 00:04:20,060
made to actually expand the range of competencies that we can quantify, that we can measure.

52
00:04:20,060 --> 00:04:24,259
When I started with PISA, which was a long time ago, I would never dream that one day

53
00:04:24,259 --> 00:04:31,250
we could assess social skills. We've done that now. Today, it's hard to imagine how

54
00:04:31,250 --> 00:04:37,910
you would assess you know things like creativity or maybe empathy things that

55
00:04:37,910 --> 00:04:42,250
we know are important but you know we should not prejudge the future

56
00:04:42,250 --> 00:04:47,129
assessments will evolve as learning will evolve and we have to become better it's

57
00:04:47,129 --> 00:04:51,589
hard to improve what you cannot see what you cannot measure so becoming better in

58
00:04:51,589 --> 00:04:57,290
this the second point I want to make about PISA is that we're not so much

59
00:04:57,290 --> 00:05:00,970
interested to look at just whether students can reproduce what they know.

60
00:05:00,970 --> 00:05:08,740
That's not the core of PISA. We're trying to look at what students

61
00:05:08,740 --> 00:05:13,240
can do with what they know. And that is an important difference, and it makes

62
00:05:13,240 --> 00:05:19,060
actually a big difference for schools in Spain in general. If we had just looked

63
00:05:19,060 --> 00:05:25,779
at students' knowledge in physics, chemistry, and biology, Spain would have

64
00:05:25,779 --> 00:05:31,660
done slightly better than it does on the current PISA test. It's not enough.

65
00:05:31,660 --> 00:05:36,639
The modern world no longer rewards you just for what you know. Google knows

66
00:05:36,639 --> 00:05:41,199
everything. The modern world rewards you really what you can do with

67
00:05:41,199 --> 00:05:45,339
what you know. The modern world rewards whether you can think like a

68
00:05:45,339 --> 00:05:50,680
scientist. We call it epistemic knowledge, epistemic understanding or conceptual

69
00:05:50,680 --> 00:05:54,879
understanding. In fact, the world of knowledge in science evolves

70
00:05:54,879 --> 00:06:02,800
very rapidly. But those enduring features are your capacity to think like a scientist,

71
00:06:02,800 --> 00:06:10,199
to design an experiment, to understand the difference between a theory and a fact. Those

72
00:06:10,199 --> 00:06:13,959
things are really, really important, and not just for the few people who become scientists

73
00:06:13,959 --> 00:06:18,300
later in their lives, but for everyone. So that's an important part of the philosophy

74
00:06:18,300 --> 00:06:24,879
of PISA. It may not be something that students usually encounter in a school test. The PISA

75
00:06:24,879 --> 00:06:27,879
The tests do look very different from a normal school test.

76
00:06:28,160 --> 00:06:31,660
There are very few kind of questions that you can just check,

77
00:06:31,959 --> 00:06:33,279
multiple choice questions.

78
00:06:33,459 --> 00:06:36,259
Most of the questions do require an engagement.

79
00:06:37,220 --> 00:06:40,720
That actually, and a quite high level of complex thinking in this,

80
00:06:40,800 --> 00:06:42,699
but that's intentionally we want to do that.

81
00:06:43,180 --> 00:06:47,300
We are also putting a great deal of effort to look at the context

82
00:06:47,300 --> 00:06:51,519
in which students learn, teachers teach, and schools operate.

83
00:06:51,519 --> 00:06:57,600
When you compare schools, you know, you can only meaningfully compare schools when you

84
00:06:57,600 --> 00:07:01,899
actually know something about the context in which schools operate.

85
00:07:01,899 --> 00:07:06,759
If you have a lot of students from disadvantaged, well, you are in a different context than

86
00:07:06,759 --> 00:07:11,180
are you in a quite wealthy neighborhood like this one here.

87
00:07:11,180 --> 00:07:15,399
Understanding that context, incorporating this into the assessment design is a very

88
00:07:15,399 --> 00:07:19,120
important ingredient of the PISA process.

89
00:07:19,120 --> 00:07:23,360
And beyond that, we're also very interested to understand something about the practices

90
00:07:23,360 --> 00:07:28,560
in the school, things like the disciplinary climate, the learning approaches, the way

91
00:07:28,560 --> 00:07:31,939
teachers work, teachers collaborate.

92
00:07:31,939 --> 00:07:36,860
All of those are very, very important features that we try to reflect because we want to

93
00:07:36,860 --> 00:07:40,560
actually explain the differences between schools.

94
00:07:40,560 --> 00:07:41,660
And you'll be amazed.

95
00:07:41,660 --> 00:07:48,120
When we started with PISA, through our statistics, we could explain about 35 percent of the variation

96
00:07:48,120 --> 00:07:54,899
of schools. Most of the variation that we observed were something that we couldn't

97
00:07:54,899 --> 00:08:02,500
explain with our kind of data. Today, we can statistically account for 85% of the variation

98
00:08:02,500 --> 00:08:08,959
of schools. The models have become very powerful. That doesn't mean that we understand the

99
00:08:08,959 --> 00:08:14,199
causal nature of those relationships, but the models have become very, very powerful

100
00:08:14,199 --> 00:08:22,379
to help us predict what makes a school succeed, what makes an education system succeed.

101
00:08:22,379 --> 00:08:26,040
The answer of how you get there is a different story, it's a complex story, and it's a story

102
00:08:26,040 --> 00:08:29,720
that is very deeply embedded in culture and tradition.

103
00:08:29,720 --> 00:08:34,559
But the factors that are the ingredients for success are pretty constant, and that's something

104
00:08:34,559 --> 00:08:37,240
that is very, very important to the PISA design.

105
00:08:37,240 --> 00:08:43,019
So an evolving instrument, something that adapts as reality changes, and something that

106
00:08:43,019 --> 00:08:49,799
tries to explain one more point on the reality you know when we tested reading

107
00:08:49,799 --> 00:08:55,559
skills in the year 2000 very hard to think back that far now but in the year

108
00:08:55,559 --> 00:09:00,259
2000 we still read books you know printed texts there was the way we

109
00:09:00,259 --> 00:09:03,779
actually read most of the time and that was actually something that involved a

110
00:09:03,779 --> 00:09:09,000
certain type of cognitive processes and a processing linear text extracting

111
00:09:09,000 --> 00:09:16,000
information that somebody else has written. That was reading. And you know when as a teacher

112
00:09:17,539 --> 00:09:20,519
you had a student who'd ask a question, you could ask that student, well, you know, look

113
00:09:20,519 --> 00:09:27,519
it up in an encyclopedia and you can actually trust the answer you find to be true, generally.

114
00:09:28,919 --> 00:09:35,919
Today your students look up something on Google and they find 20,000 different answers and

115
00:09:35,919 --> 00:09:37,360
And they have to make judgments.

116
00:09:37,600 --> 00:09:39,120
They have to navigate information.

117
00:09:39,419 --> 00:09:42,059
They have to resolve conflicting pieces of information.

118
00:09:42,379 --> 00:09:45,759
They have to build a mental representation of information

119
00:09:45,759 --> 00:09:47,600
they can't see in front of them.

120
00:09:48,240 --> 00:09:49,559
The Internet on your screen,

121
00:09:49,659 --> 00:09:53,059
you just see such a small slice of this complex world of knowledge.

122
00:09:53,639 --> 00:09:54,600
You have to navigate.

123
00:09:55,440 --> 00:09:58,500
Literacy is no longer about extracting information.

124
00:09:58,500 --> 00:10:00,519
It's about constructing information.

125
00:10:00,639 --> 00:10:02,799
It's a completely different construct.

126
00:10:04,059 --> 00:10:05,080
So what happened to PISA?

127
00:10:05,080 --> 00:10:06,940
Well, you know, Pisa did the same thing.

128
00:10:07,059 --> 00:10:11,799
In the past, you know, you get a printed text, you read it.

129
00:10:12,220 --> 00:10:14,840
Today, students have to solve the problems on a computer.

130
00:10:15,860 --> 00:10:17,860
It's more challenging for some students.

131
00:10:17,960 --> 00:10:19,840
Some of the things are more cognitively demanding,

132
00:10:19,919 --> 00:10:22,200
but that is the way in which the world changes

133
00:10:22,200 --> 00:10:24,620
and the way in which the test changes.

134
00:10:25,240 --> 00:10:27,960
Some people say, well, if you want to measure change,

135
00:10:28,120 --> 00:10:29,399
you cannot change the test.

136
00:10:30,340 --> 00:10:33,559
That's basically a very kind of conservative view

137
00:10:33,559 --> 00:10:36,820
on measuring progress, but if the world is changing

138
00:10:36,820 --> 00:10:40,399
and you don't change your test, you quickly become irrelevant.

139
00:10:41,360 --> 00:10:45,259
So actually making sure that the tests evolve is very important.

140
00:10:45,820 --> 00:10:49,559
This is basically giving you a map of the education systems

141
00:10:49,559 --> 00:10:52,259
you can compare yourself today with.

142
00:10:53,259 --> 00:10:56,460
In gray, you can see the countries of the OECD,

143
00:10:56,600 --> 00:10:59,580
the principal industrialized countries, Spain included,

144
00:10:59,799 --> 00:11:02,500
and then in blue, other countries that have joined

145
00:11:02,500 --> 00:11:05,620
the PISA assessment, and the number is becoming bigger and bigger.

146
00:11:05,620 --> 00:11:10,899
The next assessment is including – the last one we did was 72 countries, the next one

147
00:11:10,899 --> 00:11:16,000
is going to be 80 countries, and probably the one after will be in the order of 120

148
00:11:16,000 --> 00:11:17,000
countries.

149
00:11:17,000 --> 00:11:21,519
So there's actually a growing number of countries for which we have comparative data.

150
00:11:21,519 --> 00:11:23,379
Some have only very patchy data.

151
00:11:23,379 --> 00:11:29,879
Here for example you can see there are four tiny provinces in China – Beijing, Shanghai,

152
00:11:29,879 --> 00:11:34,159
Jiangsu and Guangdong, quite diverse provinces.

153
00:11:34,159 --> 00:11:36,360
Shanghai is like Madrid in Spain.

154
00:11:36,360 --> 00:11:39,960
It's a very kind of elite city, very great education system.

155
00:11:39,960 --> 00:11:44,279
If you go to Guangdong, GDP is more like Mexico.

156
00:11:44,279 --> 00:11:45,779
Huge variability in this.

157
00:11:45,779 --> 00:11:46,799
But it's only for China.

158
00:11:46,799 --> 00:11:49,700
We have only a few parts of the country covered yet.

159
00:11:49,700 --> 00:11:50,879
We are still working on that.

160
00:11:50,879 --> 00:11:53,559
And same for India and other parts of the world.

161
00:11:53,559 --> 00:11:55,019
But it's progressing.

162
00:11:55,019 --> 00:11:56,820
More and more schools and countries

163
00:11:56,820 --> 00:11:58,340
are joining the assessment.

164
00:11:58,340 --> 00:12:01,659
So you get a more and more complete picture, and you can have more choices with whom you

165
00:12:01,659 --> 00:12:03,960
can compare yourself.

166
00:12:03,960 --> 00:12:10,059
I just want to sort of show you a little bit about progress in education.

167
00:12:10,059 --> 00:12:17,220
The last assessment focused on science, very, very important set of skills today, thinking

168
00:12:17,220 --> 00:12:18,779
like a scientist.

169
00:12:18,779 --> 00:12:23,620
Many of the questions that frame ourselves around us are framed in science.

170
00:12:23,620 --> 00:12:28,120
And the first time we assessed science in depth was in 2006.

171
00:12:28,120 --> 00:12:32,820
Again, it's very hard to remember what happened in 2006.

172
00:12:32,820 --> 00:12:36,720
One of the things that you might remember is that you didn't have a smartphone yet.

173
00:12:36,720 --> 00:12:40,440
The iPhone was not yet invented.

174
00:12:40,440 --> 00:12:44,539
Twitter was still a sound by then.

175
00:12:44,539 --> 00:12:48,240
The Amazon was still a river.

176
00:12:48,240 --> 00:12:54,340
All of those kinds of developments are actually happened since the year 2006.

177
00:12:54,340 --> 00:13:00,460
But actually in the industrialized world, schools did not respond to that.

178
00:13:00,460 --> 00:13:03,700
Results and science have remained more or less as they have been.

179
00:13:03,700 --> 00:13:05,279
And the world continued to change.

180
00:13:05,279 --> 00:13:08,559
You know, maps became dynamic.

181
00:13:08,559 --> 00:13:10,820
Cars became electric.

182
00:13:10,820 --> 00:13:14,820
Cars started to drive without a driver, you know.

183
00:13:14,820 --> 00:13:19,279
We talk about virtual reality, bringing the world's most advanced knowledge in real time

184
00:13:19,279 --> 00:13:26,960
in what you do, robotics, human genetics, all of those things have dramatically evolved

185
00:13:26,960 --> 00:13:34,179
between 2009 and 2012, but actually if we sort of reflect deeply on this, schools did

186
00:13:34,179 --> 00:13:35,919
not change very much.

187
00:13:35,919 --> 00:13:40,039
What we teach in science, how we teach in science, and the learning outcomes in the

188
00:13:40,039 --> 00:13:44,779
industrialized world have been pretty stagnant over that period.

189
00:13:44,779 --> 00:13:46,919
And you know the world continues to change.

190
00:13:46,919 --> 00:13:50,100
Huge amounts of things happen in the world around us.

191
00:13:50,100 --> 00:13:56,899
Science is evolving exponentially in every aspect.

192
00:13:56,899 --> 00:13:59,220
People are developing linearly.

193
00:13:59,220 --> 00:14:01,240
We have a hard time keeping up with this change.

194
00:14:01,240 --> 00:14:07,519
And certainly if you look in education over the last decade, we have seen very, very little

195
00:14:07,519 --> 00:14:08,519
change.

196
00:14:08,519 --> 00:14:10,440
But it's not universal.

197
00:14:10,440 --> 00:14:14,740
You know, you have actually some countries that have been remarkable in their outcomes.

198
00:14:14,919 --> 00:14:19,659
You know, Portugal is a country that used to perform well below the OECD average,

199
00:14:20,940 --> 00:14:25,039
you know, was sort of the poor cousin of Europe, and now is above the OECD average.

200
00:14:26,000 --> 00:14:27,679
It's continued to progress.

201
00:14:28,460 --> 00:14:30,639
Singapore is a country that's also important, you know.

202
00:14:31,080 --> 00:14:34,419
Singapore is a top performer and doesn't stand still.

203
00:14:35,379 --> 00:14:37,279
It's also an important lesson for us, you know.

204
00:14:37,279 --> 00:14:42,120
High-performing countries are not just some stars that remain there and you can just approximate them.

205
00:14:42,580 --> 00:14:46,700
No, they also continue to improve, and actually often faster than anyone.

206
00:14:47,879 --> 00:14:53,960
The most advanced schools, the most advanced education systems keep improving very, very fast.

207
00:14:55,759 --> 00:14:57,120
And it's actually hard to do.

208
00:14:57,659 --> 00:15:00,559
If you catch up, you know, like Portugal, it's easy.

209
00:15:00,679 --> 00:15:03,600
You just look at what others are doing and try to see what you can learn from this.

210
00:15:03,899 --> 00:15:07,659
If you're at the frontier, you cannot emulate.

211
00:15:07,659 --> 00:15:12,899
way. You have to invent the future. And that's one of the reasons why, you know, the PISA

212
00:15:12,899 --> 00:15:17,620
for schools, schools like here in Madrid and around the world, we want to bring them together

213
00:15:17,620 --> 00:15:23,679
so that they can work together to invent the frontier, the next education system. If we

214
00:15:23,679 --> 00:15:27,639
can bring the most innovative teachers and school leaders together around the world,

215
00:15:27,639 --> 00:15:34,059
we can build the world's most advanced education system. So generally, limited progress, but

216
00:15:34,059 --> 00:15:39,220
some countries do show us that actually progress is possible.

217
00:15:39,220 --> 00:15:43,200
So this is basically what motivates us to look behind this, and I just want to sort

218
00:15:43,200 --> 00:15:49,059
of put countries' education systems on a map, and this is, I think, a very important point

219
00:15:49,059 --> 00:15:57,360
that Minister van Grieken made, that actually when you look at quality on the vertical axis

220
00:15:57,360 --> 00:16:03,299
and the capacity of education systems to moderate social disadvantage, equity on the horizontal

221
00:16:03,299 --> 00:16:09,000
access you can see there is no obvious trade-off everybody wants to be in the

222
00:16:09,000 --> 00:16:13,679
green area here where performance is very good and where all students succeed

223
00:16:13,679 --> 00:16:18,659
and you can see that area is not empty it's not that you know you have to

224
00:16:18,659 --> 00:16:21,759
choose between quality and equity there are actually countries all over the

225
00:16:21,759 --> 00:16:26,940
world actually that demonstrate that we can deliver strong performance for

226
00:16:26,940 --> 00:16:30,000
students from all social backgrounds there are also countries that struggle

227
00:16:30,000 --> 00:16:37,000
with both. If you are not doing well, it doesn't mean that you're equitable. Actually, it can

228
00:16:37,620 --> 00:16:43,779
be difficult for both areas. There are countries in the top left corner where you can say they

229
00:16:43,779 --> 00:16:49,500
do quite well on average, but you can see important social disparities. I told you how

230
00:16:49,500 --> 00:16:54,120
great Singapore is as an education system overall, but actually social background makes

231
00:16:54,120 --> 00:16:59,419
quite a big difference, even in Singapore. And then there are countries that don't do

232
00:16:59,419 --> 00:17:03,799
so well but come out quite equitably. But really the important point is that we should

233
00:17:03,799 --> 00:17:10,400
not give up to achieve excellence and equity at the very same time. It's possible for countries

234
00:17:10,400 --> 00:17:15,279
and it's possible for schools. Same thing that we see among countries as what we see

235
00:17:15,279 --> 00:17:21,480
among schools. Some schools have an amazing capacity to attract the most talented teachers

236
00:17:21,480 --> 00:17:26,720
in the most challenging classrooms and to make sure that every student benefits from

237
00:17:26,720 --> 00:17:34,160
excellent learning. Let's have a look at this in another way. This is actually, you know,

238
00:17:34,160 --> 00:17:39,000
some people they look at this chart and say, well, you know, this is all about culture

239
00:17:39,000 --> 00:17:47,210
and context. How can you actually change those things? So in this chart, I'm going to take

240
00:17:47,210 --> 00:17:53,309
the culture and the context out. I compare apples with apples. I compare students across

241
00:17:53,309 --> 00:17:58,890
countries who have the same family background. Have a look at this. I start with the Dominican

242
00:17:58,890 --> 00:18:05,970
Republic. In the red square, I show you students from the 10% poorest families internationally.

243
00:18:05,970 --> 00:18:11,890
In the green triangle, you can see the students from the 10% wealthiest families internationally,

244
00:18:11,890 --> 00:18:16,809
how they do in the Dominican Republic. And what you see is that there's a huge achievement

245
00:18:16,809 --> 00:18:21,890
gap. You come from a poor family, you do really poorly in the Dominican Republic. You come

246
00:18:21,890 --> 00:18:23,589
from the wealthy family, you do okay.

247
00:18:25,470 --> 00:18:27,430
Some people who see that say, well, you know,

248
00:18:27,509 --> 00:18:28,390
that's what we told you.

249
00:18:28,549 --> 00:18:29,490
Poverty is destiny.

250
00:18:30,329 --> 00:18:31,730
There's nothing we can do about it.

251
00:18:32,650 --> 00:18:35,190
But when you actually see how this plays out

252
00:18:35,190 --> 00:18:37,670
in different countries, you can see that the same student

253
00:18:37,670 --> 00:18:41,450
with the same social background and the same family income

254
00:18:41,450 --> 00:18:43,349
and the same employment of the parents

255
00:18:43,349 --> 00:18:47,369
have so different performance levels across countries.

256
00:18:50,079 --> 00:18:51,660
The most interesting one is this.

257
00:18:51,660 --> 00:18:56,279
You know, look at the 10% of the poorest children in Vietnam.

258
00:18:57,920 --> 00:19:01,240
They come from, you know, really difficult family backgrounds.

259
00:19:02,279 --> 00:19:05,380
And they do as well as the average student in the OECD area.

260
00:19:06,259 --> 00:19:10,880
And they do better than the 10% wealthiest students in some other countries.

261
00:19:12,789 --> 00:19:15,990
It's a powerful illustration that poverty need not be destiny,

262
00:19:16,130 --> 00:19:21,529
that education systems can do well in order to moderate social disadvantage.

263
00:19:22,289 --> 00:19:24,289
Of course, it doesn't happen by chance.

264
00:19:24,450 --> 00:19:27,309
If you actually go to visit schools in Vietnam, you'll be amazed.

265
00:19:27,309 --> 00:19:29,450
You go to a rural community in Vietnam,

266
00:19:29,930 --> 00:19:32,509
and you're going to see the school is in a fantastic condition.

267
00:19:32,630 --> 00:19:34,869
It has great principals and great teachers.

268
00:19:36,109 --> 00:19:41,569
And you ask yourself, how did they succeed to get those very talented teachers

269
00:19:41,569 --> 00:19:42,990
into those difficult schools?

270
00:19:44,009 --> 00:19:47,509
And the answer is, that's how every career path goes.

271
00:19:47,509 --> 00:20:00,569
If you are a vice principal in a high-performing school like this one and you say, I want to become principal, well, the education system will tell you, it's a great ambition, but please first help us turn around one of the lowest-performing schools.

272
00:20:00,849 --> 00:20:02,329
Go to teach in a rural community.

273
00:20:02,809 --> 00:20:05,250
And if it works, actually, you're going to have a great career.

274
00:20:06,029 --> 00:20:07,470
And that's what they do with every teacher.

275
00:20:07,569 --> 00:20:08,789
That's what they do with every school.

276
00:20:08,869 --> 00:20:12,269
And that's how they leverage the potential of disadvantaged children.

277
00:20:13,269 --> 00:20:17,369
And what's so interesting, this is not only mirrored in the PISA results.

278
00:20:17,509 --> 00:20:21,509
It's also mirrored in the mindset of students.

279
00:20:21,509 --> 00:20:25,509
Once we ask students a question about mathematics,

280
00:20:25,509 --> 00:20:26,509
we ask them, you know,

281
00:20:26,509 --> 00:20:30,509
what do you believe makes you successful in mathematics?

282
00:20:30,509 --> 00:20:36,140
We call this self-efficacy, but that's sort of the idea of it.

283
00:20:36,140 --> 00:20:39,140
And then, you know, the majority of students in Spain said

284
00:20:39,140 --> 00:20:42,140
the answer is very simple, you know, it's about talent.

285
00:20:42,140 --> 00:20:45,140
If I'm not a genius, I'm not going to be studying mathematics

286
00:20:45,140 --> 00:20:48,140
and I'm going to do something else.

287
00:20:48,140 --> 00:20:53,799
So, they believe, you know, that success is something that is outside their control.

288
00:20:53,799 --> 00:20:57,440
In the education system, schools will just source it.

289
00:20:57,440 --> 00:21:04,579
If you ask that same question to a student in, you know, Vietnam, China, Singapore, nine

290
00:21:04,579 --> 00:21:09,819
out of ten students tell you, if I try really, really hard, I trust my teacher is going to

291
00:21:09,819 --> 00:21:12,140
help me and I'm going to be successful.

292
00:21:12,140 --> 00:21:16,660
So, the education system tells those students every day, you know, it depends on you.

293
00:21:16,660 --> 00:21:21,940
If you put the effort in, we will invest whatever it takes to help you to succeed.

294
00:21:21,940 --> 00:21:25,460
And the schools have the time and resources for this.

295
00:21:25,460 --> 00:21:29,819
Teachers spend many hours every day to work individually with students from disadvantaged

296
00:21:29,819 --> 00:21:34,079
backgrounds, struggling students, to support and to help them.

297
00:21:34,079 --> 00:21:36,559
And I'm going to come in a moment to this, how they can do this.

298
00:21:36,559 --> 00:21:39,960
You know, you ask yourself how can teachers afford this.

299
00:21:39,960 --> 00:21:42,819
Teachers work very closely with parents.

300
00:21:42,819 --> 00:21:48,019
strong engagement between schools and community because this is part of the teacher's kind

301
00:21:48,019 --> 00:21:53,000
of role in those countries. Well, you know, here I just want to show you the bottom line.

302
00:21:53,000 --> 00:21:59,519
The bottom line is really that the world's most disadvantaged children can achieve world

303
00:21:59,519 --> 00:22:04,839
class standards if the conditions are right. This is not just about education. I'm not

304
00:22:04,839 --> 00:22:09,200
saying that. It's not just about schools. It's about the whole context. But it's something

305
00:22:09,200 --> 00:22:15,019
I think the world can clearly learn from a lot.

306
00:22:15,019 --> 00:22:20,539
So far I've talked about learning outcomes, but I want to look at something different.

307
00:22:20,539 --> 00:22:24,440
We ask students also, what do you want to do in your life?

308
00:22:24,440 --> 00:22:30,599
We know how well you did on the science test, but do you want to become a scientist?

309
00:22:30,599 --> 00:22:33,299
And actually the answer is different.

310
00:22:33,299 --> 00:22:39,559
This is the share of students who said, at age 30 I want to become a scientist, in human

311
00:22:39,559 --> 00:22:45,799
science and any kind of sciences and you can see it varies a lot and you just a minute ago i told

312
00:22:45,799 --> 00:22:51,319
you you know how great countries like japan korea vietnam china and finland maybe germany are in

313
00:22:51,319 --> 00:22:58,279
terms of science learning outcomes but as you can see on the chat the students don't want to become

314
00:22:58,279 --> 00:23:03,960
scientists something has gone wrong in this education system they learned something but

315
00:23:03,960 --> 00:23:08,380
but they lost that aspiration to actually move into science.

316
00:23:08,380 --> 00:23:11,500
Well, if you look to students in Spain,

317
00:23:11,500 --> 00:23:14,680
they are scoring much higher on this.

318
00:23:14,680 --> 00:23:17,039
Many students in Spain, and even the United States

319
00:23:17,039 --> 00:23:18,960
is another good example.

320
00:23:18,960 --> 00:23:22,859
The United States doesn't do well on the PISA science test,

321
00:23:22,859 --> 00:23:25,619
but the students aspire to become

322
00:23:25,619 --> 00:23:29,000
a scientist or an engineer.

323
00:23:29,000 --> 00:23:32,920
So now you look at this chart, and maybe you ask yourself,

324
00:23:32,920 --> 00:23:34,579
oh, perhaps we have to choose.

325
00:23:34,579 --> 00:23:38,079
Either our students learn something about science,

326
00:23:38,079 --> 00:23:41,480
or they want to become scientists.

327
00:23:41,480 --> 00:23:45,859
Let's have a look at this in a little bit more detail.

328
00:23:45,859 --> 00:23:47,480
One of the things that we have seen

329
00:23:47,480 --> 00:23:55,420
is that the big moderator that translates better performance

330
00:23:55,420 --> 00:23:57,779
and better aspirations has something

331
00:23:57,779 --> 00:24:02,660
to do with enjoyment, students liking science.

332
00:24:02,660 --> 00:24:05,900
You can see on this chart, if you do not enjoy science,

333
00:24:05,900 --> 00:24:08,099
Science is just a boring school subject for you.

334
00:24:08,099 --> 00:24:11,779
You may still do well on the test, particularly true for girls.

335
00:24:12,299 --> 00:24:13,779
They do quite well on the test.

336
00:24:14,180 --> 00:24:19,359
But you can see on the chart here that it doesn't translate into better learning outcomes,

337
00:24:19,420 --> 00:24:20,539
or only marginally.

338
00:24:20,700 --> 00:24:23,140
If you do better on the test, you move up here,

339
00:24:23,819 --> 00:24:26,940
only a little bit gain on the kind of career aspiration.

340
00:24:28,200 --> 00:24:32,220
But where students really like science, they enjoy science,

341
00:24:32,839 --> 00:24:34,599
you can see the picture is totally different.

342
00:24:34,599 --> 00:24:39,720
There you can see basically UN students, you know, see science as something that opens

343
00:24:39,720 --> 00:24:41,640
life opportunities that they enjoy.

344
00:24:41,640 --> 00:24:47,700
You can see suddenly better performance actually means better career aspirations.

345
00:24:47,700 --> 00:24:49,740
So that's the missing link.

346
00:24:49,740 --> 00:24:55,920
There's performance, there's career aspirations, but those are moderated through something

347
00:24:55,920 --> 00:25:00,460
that has to do with effective dimension, social and emotional components.

348
00:25:00,460 --> 00:25:04,920
And it tells us, you know, if you're a science teacher and you're not getting that right,

349
00:25:04,920 --> 00:25:10,819
you're not getting your students really interested in the subject, well, you may still get good

350
00:25:10,819 --> 00:25:15,640
science results, but your students will not become scientists.

351
00:25:15,640 --> 00:25:17,980
So you can group countries like this.

352
00:25:17,980 --> 00:25:22,960
In red, I'm going to put countries that do well on the scientists.

353
00:25:22,960 --> 00:25:28,680
Here in violet, I'm going to put countries where students believe in science to solve

354
00:25:28,680 --> 00:25:33,539
social problems sort of their trust in scientific methods and in blue I put

355
00:25:33,539 --> 00:25:38,819
those countries that have career aspirations in science of course you

356
00:25:38,819 --> 00:25:44,339
want to be in the middle can we get everything right and the answer is

357
00:25:44,339 --> 00:25:49,539
actually some countries do Singapore is a great example you know they did well

358
00:25:49,539 --> 00:25:55,140
on the PISA test students trust scientific method as students want to

359
00:25:55,140 --> 00:25:57,160
become scientists.

360
00:25:57,160 --> 00:25:59,680
Canada, Slovenia, Australia, United Kingdom,

361
00:25:59,680 --> 00:26:01,859
and at a lower level, also Ireland and Portugal.

362
00:26:01,859 --> 00:26:05,940
So it is actually possible to combine

363
00:26:05,940 --> 00:26:08,720
the cognitive, the social, and emotional factors that

364
00:26:08,720 --> 00:26:10,579
drive science outcomes.

365
00:26:10,579 --> 00:26:13,700
You look at Chinese Taipei, Hong Kong, New Zealand, Denmark,

366
00:26:13,700 --> 00:26:17,380
and there you can see students do well on the science test.

367
00:26:17,380 --> 00:26:20,099
They also trust in science, but they

368
00:26:20,099 --> 00:26:23,339
don't want to do it themselves.

369
00:26:23,339 --> 00:26:25,960
Here you have a whole list of countries,

370
00:26:25,960 --> 00:26:31,240
And I mentioned some already where you can say they do really well on the science test,

371
00:26:31,240 --> 00:26:34,200
but students don't believe in science and they don't want to become scientists.

372
00:26:34,200 --> 00:26:40,480
There is something important missing in those education systems, and we only see it if we

373
00:26:40,480 --> 00:26:41,480
look at it.

374
00:26:41,480 --> 00:26:44,460
If you only look at the scores, you're not going to see it.

375
00:26:44,460 --> 00:26:48,359
And then there are some countries like Spain where you can say students generally believe

376
00:26:48,359 --> 00:26:54,660
in science, they are very positive about scientific careers, but the schools do not prepare them

377
00:26:54,660 --> 00:27:00,220
well enough for it. The red part is missing. And then you have basically countries where

378
00:27:00,220 --> 00:27:04,819
you can see sort of there's only one of those two remaining components there. But the important

379
00:27:04,819 --> 00:27:10,579
point is, you know, we need to look at science in a quite holistic way to get a good comprehensive

380
00:27:10,579 --> 00:27:15,279
assessment of what is good. And I think the interesting part also here is that it's not

381
00:27:15,279 --> 00:27:21,839
an either or. Like excellence and equity, we do not have to sort of contrast, you know.

382
00:27:21,839 --> 00:27:26,440
Some people believe, well, if you want to do well on science, you just have to learn

383
00:27:26,440 --> 00:27:28,599
and work hard, and it's not fun.

384
00:27:28,599 --> 00:27:30,740
It's not true.

385
00:27:30,740 --> 00:27:35,359
Actually you can make science teaching fun and get good results.

386
00:27:35,359 --> 00:27:38,710
Not easy, but it's possible.

387
00:27:38,710 --> 00:27:42,950
I want to sort of talk about something that is really sort of touching on a very difficult

388
00:27:42,950 --> 00:27:50,890
issue, and that's basically about learning outcomes, how they relate to what we do.

389
00:27:50,890 --> 00:27:56,130
If in Madrid you teach one hour more science, you're going to get better science outcomes.

390
00:27:56,130 --> 00:27:59,450
If you teach one hour more mathematics, you're going to get better math outcomes.

391
00:27:59,450 --> 00:28:04,230
If you teach one hour history more, you get better history outcomes.

392
00:28:04,230 --> 00:28:10,009
Within a school, within a country, there's a clear, positive relationship between time

393
00:28:10,009 --> 00:28:11,009
and outcomes.

394
00:28:11,009 --> 00:28:12,650
Time really matters.

395
00:28:12,650 --> 00:28:14,369
That's why we have so much competition.

396
00:28:14,369 --> 00:28:17,509
Every teacher wants to have their subject being more.

397
00:28:17,509 --> 00:28:20,529
It really matters.

398
00:28:20,529 --> 00:28:25,069
If I look at this across countries, you'll be surprised.

399
00:28:25,069 --> 00:28:26,230
It looks like this.

400
00:28:26,230 --> 00:28:32,490
On the horizontal axis, you have the amount of time that students, on average, spend learning.

401
00:28:32,490 --> 00:28:35,009
And on the vertical axis, their performance.

402
00:28:35,009 --> 00:28:38,569
You look at this chart, and it looks like the more students learn, the more time they

403
00:28:38,569 --> 00:28:43,630
spend in school in a country, the worse they come out in PISA.

404
00:28:43,630 --> 00:28:45,730
Isn't that the opposite I just said?

405
00:28:45,730 --> 00:28:48,809
More time produces better outcomes?

406
00:28:48,809 --> 00:28:53,490
In a country, more time produces better outcomes, and across countries, more time makes things

407
00:28:53,490 --> 00:28:55,920
worse.

408
00:28:55,920 --> 00:28:59,359
How do you resolve that?

409
00:28:59,359 --> 00:29:03,819
Actually the answer is quite straightforward.

410
00:29:03,819 --> 00:29:11,319
Learning outcomes are always the product of the quantity of learning, the hours you spend,

411
00:29:11,319 --> 00:29:14,420
and the quality of instruction.

412
00:29:14,420 --> 00:29:16,740
Both things matter.

413
00:29:16,740 --> 00:29:18,940
And you can see that very nicely on the next chart.

414
00:29:18,940 --> 00:29:23,480
Here I show you basically in blue the amount of time that students spend in school and

415
00:29:23,480 --> 00:29:25,140
it varies across countries.

416
00:29:25,140 --> 00:29:29,680
In yellow I show you things like homework and tutoring, lots of things that happen in

417
00:29:29,680 --> 00:29:32,599
terms of learning out of school.

418
00:29:32,599 --> 00:29:38,559
And so the total length tells you the amount of time that students spend learning.

419
00:29:38,559 --> 00:29:46,039
But I can now also show you the value that is created per hour of learning.

420
00:29:46,039 --> 00:29:49,759
And you can see that varies hugely across countries.

421
00:29:49,759 --> 00:29:54,539
If you go to countries on the left side, like Finland is the number one here, Germany, Switzerland,

422
00:29:54,539 --> 00:30:00,759
and Japan, there you can really see that students don't spend that much time learning at home

423
00:30:00,759 --> 00:30:06,720
or learning at school, but every hour creates huge value.

424
00:30:06,720 --> 00:30:12,279
You go to the right-hand of the chart, and you can see actually, you know, may not just

425
00:30:12,279 --> 00:30:18,640
be a little time, but often learning gains are very limited. This tells us that, you

426
00:30:18,640 --> 00:30:24,519
know, the productivity in education, the value that is created per hour, varies hugely across

427
00:30:24,519 --> 00:30:29,339
countries. It's also true, you know, China is an interesting part, you know. Overall,

428
00:30:29,500 --> 00:30:34,460
the results in China are very good, you know. Why are they good? Because students work 60

429
00:30:34,460 --> 00:30:39,240
hours, you know. More than adults are allowed to work, you know. Students work more than

430
00:30:39,240 --> 00:30:45,779
60 hours, or about 60 hours. But if you look actually per hour of learning, actually China

431
00:30:45,779 --> 00:30:51,640
is not so great. It's very important for us not only to look at the volume of learning

432
00:30:51,640 --> 00:30:58,660
time, but also the quality of instruction. Both can vary. And that explains the relationship

433
00:30:58,660 --> 00:31:04,140
across countries. Some countries achieve very good results with limited time, and other

434
00:31:04,140 --> 00:31:10,299
countries don't achieve very good results despite lots of time. It's important to look

435
00:31:10,299 --> 00:31:13,940
at the quality of learning and the quantity of learning.

436
00:31:14,099 --> 00:31:16,359
Of course, you know, quantity always helps.

437
00:31:16,980 --> 00:31:20,339
If you teach the same style, adding one more hour,

438
00:31:20,440 --> 00:31:21,339
you get better outcomes.

439
00:31:22,119 --> 00:31:26,039
But often, you know, schools are much more better off

440
00:31:26,039 --> 00:31:28,140
if they can actually change the way they teach,

441
00:31:28,279 --> 00:31:30,519
change the way students learn.

442
00:31:31,160 --> 00:31:33,420
Because the only scarce resource in education

443
00:31:33,420 --> 00:31:35,480
is student learning time.

444
00:31:36,059 --> 00:31:38,700
You can always add more money to education.

445
00:31:38,700 --> 00:31:40,640
is always going to be a good investment.

446
00:31:41,539 --> 00:31:43,299
Money and education is always well spent.

447
00:31:44,539 --> 00:31:49,140
But what you can't do is extend the hour, the day, beyond 24 hours.

448
00:31:49,279 --> 00:31:51,460
So there are limits to what we can do with students,

449
00:31:51,619 --> 00:31:53,720
and using the time of students well,

450
00:31:54,079 --> 00:31:57,920
because also students want to have a good work-life balance, is important.

451
00:31:59,220 --> 00:32:04,200
I should also say that time in school usually counts more than time on homework.

452
00:32:04,880 --> 00:32:05,720
That's very clear.

453
00:32:05,720 --> 00:32:09,319
the amount of time that students spend in school

454
00:32:09,319 --> 00:32:12,059
is more closely related to learning outcomes

455
00:32:12,059 --> 00:32:14,279
than the amount of time that students spend out of school

456
00:32:14,279 --> 00:32:16,700
but both are very important

457
00:32:16,700 --> 00:32:20,099
I often hear people criticizing homework

458
00:32:20,099 --> 00:32:22,960
but actually if we want students to become independent learners

459
00:32:22,960 --> 00:32:26,339
students who can basically regulate their own learning processes

460
00:32:26,339 --> 00:32:29,400
and so on, then homework is a very important ingredient

461
00:32:29,400 --> 00:32:31,539
to get there, so it fulfills the purpose

462
00:32:31,539 --> 00:32:35,220
even if its level of productivity in the school sense

463
00:32:35,220 --> 00:32:38,440
is not so high, always.

464
00:32:38,440 --> 00:32:43,559
Last point I want to make on this is really about governance.

465
00:32:43,559 --> 00:32:46,059
One of the things that our data show very clearly

466
00:32:46,059 --> 00:32:50,140
where schools are involved in setting

467
00:32:50,140 --> 00:32:53,619
the direction for their school, working on staff management,

468
00:32:53,619 --> 00:32:56,420
human resources, all of those kinds of things,

469
00:32:56,420 --> 00:32:59,720
we do generally see better outcomes.

470
00:32:59,720 --> 00:33:01,680
Same for teachers.

471
00:33:01,680 --> 00:33:04,940
Teachers are very important, the kind of responsibility

472
00:33:04,940 --> 00:33:11,140
that rests on teachers for managing this affairs is a very important predictor for outcomes.

473
00:33:11,140 --> 00:33:17,420
And that's an area where, you know, even in Madrid, you know, many schools in high-performing

474
00:33:17,420 --> 00:33:20,059
education systems have much greater responsibilities.

475
00:33:20,059 --> 00:33:26,220
You know, in some countries schools have to, you know, decide whom they hire, decide how

476
00:33:26,220 --> 00:33:31,720
they pay their teachers, decide on the way they design their curriculum, very complex

477
00:33:31,720 --> 00:33:35,920
human resource management decisions resting on the shoulders of schools and

478
00:33:35,920 --> 00:33:41,619
of teachers as well and that's the way I mentioned this already how different

479
00:33:41,619 --> 00:33:48,099
countries make different spending choices if you compare for example

480
00:33:48,099 --> 00:33:56,220
Madrid with Shanghai the student staff ratio in Madrid and in Shanghai is about

481
00:33:56,220 --> 00:34:01,859
the same you spend about the same number of teachers for every group of students

482
00:34:01,859 --> 00:34:03,400
between Madrid and Shanghai.

483
00:34:04,579 --> 00:34:07,579
But the class size in Shanghai

484
00:34:07,579 --> 00:34:11,440
is almost twice as large as in Madrid.

485
00:34:13,440 --> 00:34:15,880
And now you ask yourself, how can that be?

486
00:34:16,139 --> 00:34:17,800
You know, we have the same number of teachers

487
00:34:17,800 --> 00:34:20,719
and a similar sort of proportion relative to students.

488
00:34:21,360 --> 00:34:23,719
And in Madrid, the class size is half

489
00:34:23,719 --> 00:34:24,920
than the class size in Shanghai.

490
00:34:25,860 --> 00:34:26,840
What does it mean?

491
00:34:26,900 --> 00:34:29,639
What do the teachers do in Shanghai if they don't teach?

492
00:34:29,639 --> 00:34:31,900
and the answer is

493
00:34:31,900 --> 00:34:34,800
because the class size is twice

494
00:34:34,800 --> 00:34:35,880
as in Madrid

495
00:34:35,880 --> 00:34:38,179
teachers only teach half the hours

496
00:34:38,179 --> 00:34:40,559
depending on your status

497
00:34:40,559 --> 00:34:43,519
teachers teach between 11 and 16 hours per week

498
00:34:43,519 --> 00:34:46,099
and now you say

499
00:34:46,099 --> 00:34:47,079
oh these lucky teachers

500
00:34:47,079 --> 00:34:48,420
but actually they work more

501
00:34:48,420 --> 00:34:49,659
than teachers in Madrid

502
00:34:49,659 --> 00:34:52,719
or about the same number of hours

503
00:34:52,719 --> 00:34:55,219
so what do they do in the hours

504
00:34:55,219 --> 00:34:56,480
in which they don't teach

505
00:34:56,480 --> 00:34:58,239
well you know

506
00:34:58,239 --> 00:35:00,280
They spend a lot of time working with their colleagues

507
00:35:00,280 --> 00:35:02,420
to prepare lessons, to analyze lessons.

508
00:35:02,539 --> 00:35:03,840
They would actually once per week

509
00:35:03,840 --> 00:35:05,360
observe somebody else's classroom.

510
00:35:05,679 --> 00:35:07,960
Once per month, they take part in a teacher competition.

511
00:35:08,300 --> 00:35:10,079
Once per year, they walk in the province.

512
00:35:11,400 --> 00:35:15,480
They will call the parents of the students once per week.

513
00:35:16,119 --> 00:35:17,340
Very, very intensive.

514
00:35:17,340 --> 00:35:19,539
They spend about eight hours per week

515
00:35:19,539 --> 00:35:21,460
with students individually

516
00:35:21,460 --> 00:35:23,519
to help struggling students to succeed.

517
00:35:24,940 --> 00:35:27,639
So what you can see here is that two cities

518
00:35:27,639 --> 00:35:30,380
spend the same amount of money,

519
00:35:31,619 --> 00:35:32,659
spend the same resources,

520
00:35:33,119 --> 00:35:35,340
but the way they use their resources

521
00:35:35,340 --> 00:35:36,920
varies hugely across countries.

522
00:35:37,900 --> 00:35:39,059
In some countries,

523
00:35:39,380 --> 00:35:40,619
there's a lot of responsibility

524
00:35:40,619 --> 00:35:42,500
for resource management on the school.

525
00:35:42,960 --> 00:35:43,760
In other countries,

526
00:35:44,099 --> 00:35:46,199
all of that is more centrally done.

527
00:35:46,280 --> 00:35:48,179
So it's very important for us to look at this.

528
00:35:49,159 --> 00:35:51,059
Now, the final part,

529
00:35:51,059 --> 00:35:53,000
I want to devote actually more closely

530
00:35:53,000 --> 00:35:55,380
to the PISA 4 schools assessment.

531
00:35:56,179 --> 00:35:57,280
What I show you in this

532
00:35:57,280 --> 00:36:02,519
is basically a picture on the vertical axis you see the performance of

533
00:36:02,519 --> 00:36:07,539
individual schools in Spain and on the horizontal axis you see the social

534
00:36:07,539 --> 00:36:12,760
background of schools now and what you can see is that actually as the social

535
00:36:12,760 --> 00:36:17,380
background becomes more privileged you know as you move from to the right here

536
00:36:17,380 --> 00:36:21,519
you can see that schools generally do better this is basically telling you

537
00:36:21,519 --> 00:36:26,860
that on average sort of more privileged schools come out better just for fun

538
00:36:26,860 --> 00:36:31,119
I've also separated the schools between the north and the south of Spain now

539
00:36:31,119 --> 00:36:34,659
because some people say well you know in the north of Spain the school systems

540
00:36:34,659 --> 00:36:38,860
are a lot better than in the south of Spain actually you know when you look at

541
00:36:38,860 --> 00:36:43,239
this in relation to a social background similar schools come out quite similarly

542
00:36:43,239 --> 00:36:48,400
now the difference is largely accounted for by social background now you can see

543
00:36:48,400 --> 00:36:52,739
basically if you look at the yellow and this blue school here they come out more

544
00:36:52,739 --> 00:36:56,699
all is the same. So it's a story of social disadvantage that you can see here.

545
00:36:56,840 --> 00:37:00,579
But that's the overall pattern. The interesting feature comes when you look at this

546
00:37:00,579 --> 00:37:04,360
actually, you know, you now look at individual schools. Have a look at this.

547
00:37:05,039 --> 00:37:08,320
I take two schools coming from the same neighborhood,

548
00:37:08,579 --> 00:37:12,719
same social neighborhood. Actually one blue one

549
00:37:12,719 --> 00:37:16,820
and a yellow one. And you can see, despite the fact

550
00:37:16,820 --> 00:37:20,860
that they have children from the same family backgrounds, they come out

551
00:37:20,860 --> 00:37:30,019
both formulae here being at one standard deviation above the mean, they are 100 PISA points apart.

552
00:37:30,019 --> 00:37:38,050
Now what does 100 PISA points mean? Almost three school years. Huge performance difference

553
00:37:38,050 --> 00:37:44,230
between two schools with the same kind of family background. And that's sort of something

554
00:37:44,230 --> 00:37:48,289
what the power of school level comparisons are. They actually show us that something

555
00:37:48,289 --> 00:37:53,869
is different. And you can basically see that that is the idea why we started this PISA

556
00:37:53,869 --> 00:37:57,849
for Schools comparison. The big differences don't lie between countries, they don't

557
00:37:57,849 --> 00:38:02,090
lie between regions, they don't lie between the north and the south of countries, they

558
00:38:02,090 --> 00:38:08,010
actually lie between individual schools. And that's why we devote a lot of attention

559
00:38:08,010 --> 00:38:13,429
to actually build tests for individual schools, look at the results, and give the results

560
00:38:13,429 --> 00:38:18,110
to schools we don't compare the results now we don't basically one of the things

561
00:38:18,110 --> 00:38:22,550
that we decided not to do is Peter for school to store you know provide a list

562
00:38:22,550 --> 00:38:26,690
of schools ranking of schools some are good some this not the purpose the

563
00:38:26,690 --> 00:38:32,210
purpose is really to give every school its own tools you can see how you do and

564
00:38:32,210 --> 00:38:37,130
how you compare with similar schools or different schools this is philosophy

565
00:38:37,130 --> 00:38:41,550
this is the idea of this assessment it's not an outside look into the school but

566
00:38:41,550 --> 00:38:45,949
But it's basically a look inside the school to the outside world.

567
00:38:45,949 --> 00:38:48,050
Now that's basically the idea.

568
00:38:48,050 --> 00:38:50,690
And the test is comparable.

569
00:38:50,690 --> 00:38:56,130
We use a slightly simpler test for the schools because typically they don't want to spend

570
00:38:56,130 --> 00:39:01,230
so much time on this as we do in the regular PISA test where we sort of have huge populations

571
00:39:01,230 --> 00:39:03,469
and large sets of items.

572
00:39:03,469 --> 00:39:08,789
So it's a little bit simpler version, but we can compare the results reasonably well.

573
00:39:08,789 --> 00:39:15,670
So basically, you can see in this information of how well your students perform relative

574
00:39:15,670 --> 00:39:21,510
to other schools, relative to similar schools, relative to other countries, relative to other

575
00:39:21,510 --> 00:39:26,030
education systems, there are many comparisons you can make.

576
00:39:26,030 --> 00:39:33,030
You can also see, learn something about the learning environment, discipline, school climate,

577
00:39:33,030 --> 00:39:38,869
All of the things that we actually measure and that you find reflected in your report.

578
00:39:38,869 --> 00:39:44,010
And so you can start, for example, here somewhere in Spain, and then you can see, you know,

579
00:39:44,010 --> 00:39:48,789
how do you compare against other regions in Spain in a school.

580
00:39:48,789 --> 00:39:53,070
And you can say, well, now I'm actually also interested in how schools in England compare,

581
00:39:53,070 --> 00:39:55,590
and you can actually see that kind of comparisons.

582
00:39:55,590 --> 00:40:01,909
Or you say, well, let's see how my school compares between Madrid and New York.

583
00:40:01,909 --> 00:40:06,949
as well. And the same is true when you look at high performing education systems. How

584
00:40:06,949 --> 00:40:13,190
does my school compare against a school in Shanghai, a school in Japan, a school in Finland,

585
00:40:13,190 --> 00:40:19,010
and so on. So that's basically the idea, to give schools the tools to see themselves in

586
00:40:19,010 --> 00:40:23,289
a global perspective, in a comparative perspective.

587
00:40:23,289 --> 00:40:28,510
The assessment so far is focused, as I said at the beginning, on reading, math and science.

588
00:40:28,510 --> 00:40:33,750
We will be making the assessment of collaborative problem-solving skills also available as part

589
00:40:33,750 --> 00:40:34,750
of the next round.

590
00:40:34,750 --> 00:40:36,989
That's something that we've just developed for PISA.

591
00:40:36,989 --> 00:40:41,230
We have the student questionnaires looking at the social demographic features, and we

592
00:40:41,230 --> 00:40:47,670
have the questionnaire for schools that basically looks at the characteristics of schools.

593
00:40:47,670 --> 00:40:50,829
And that's basically sort of a timeline.

594
00:40:50,829 --> 00:40:58,510
We piloted this in 2013-14, and so since 2015, all of this is now available to basically

595
00:40:58,510 --> 00:41:03,510
any school in Spain that wants to compare itself in this framework.

596
00:41:03,510 --> 00:41:04,889
And you're not the only country.

597
00:41:04,889 --> 00:41:08,969
The United States actually was the country starting that first.

598
00:41:08,969 --> 00:41:15,050
Russia came into the picture quite late, but actually has all of the schools of the city

599
00:41:15,050 --> 00:41:21,369
of Moscow have taken part in this, England and Wales, then Brunei, another exotic country,

600
00:41:21,369 --> 00:41:24,510
and then the United Arab Emirates and so on.

601
00:41:24,510 --> 00:41:30,150
There are countries that have dedicated time and resources to this.

602
00:41:30,150 --> 00:41:35,909
You can see the results of the school, you can think about what you want to do, comparing

603
00:41:35,909 --> 00:41:38,889
yourself and then plan those kinds of improvements.

604
00:41:38,889 --> 00:41:42,369
In fact, some schools have gone very far in this.

605
00:41:42,369 --> 00:41:47,429
Some schools in the neighborhood of Washington in the United States have built a whole professional

606
00:41:47,429 --> 00:41:51,849
development program around practices that they found in other countries.

607
00:41:51,849 --> 00:41:55,550
They were actually, you know, something that you will know, that the world knows quite

608
00:41:55,550 --> 00:41:59,789
well that, you know, in a country like Singapore, every school has a professional learning community

609
00:41:59,789 --> 00:42:04,130
where teachers would regularly meet with the principal to discuss, you know, pedagogical

610
00:42:04,130 --> 00:42:06,269
practice, all of those kinds of things.

611
00:42:06,269 --> 00:42:07,809
In Washington, they were totally surprised.

612
00:42:07,809 --> 00:42:11,349
They didn't know this exists, so they said, well, let's learn from this.

613
00:42:11,349 --> 00:42:15,429
work with them they build a whole program around practices in other

614
00:42:15,429 --> 00:42:21,309
countries the last point I really want to make is here you have an example from

615
00:42:21,309 --> 00:42:26,590
pizza for schools here you take a very high-performing school in the case of

616
00:42:26,590 --> 00:42:31,170
Spain and the red dog now and you look the result is great but you look at the

617
00:42:31,170 --> 00:42:36,309
social background and you say also while the school actually should be great it

618
00:42:36,309 --> 00:42:40,030
is pretty much on the predicted line a little bit better than what you predict

619
00:42:40,030 --> 00:42:44,650
that's the blue line a little bit better but sort of more or less where you

620
00:42:44,650 --> 00:42:48,989
expect it to be and that's the perspective that you wouldn't get if we

621
00:42:48,989 --> 00:42:51,929
simply would you know give you the performance results only without

622
00:42:51,929 --> 00:42:57,809
actually contextualizing performance in the social demographic context of schools

623
00:42:57,809 --> 00:43:02,230
here you know I compare your school here I compared your schools with schools in

624
00:43:02,230 --> 00:43:08,170
Spain now but now I can compare your schools with schools in Finland actually

625
00:43:08,170 --> 00:43:13,210
you can see that here very nicely if you look at actually schools in France and

626
00:43:13,210 --> 00:43:21,190
Finland the schools and France are the ones that basically participated in the

627
00:43:21,190 --> 00:43:26,050
2015 test and you can see actually in France if you come from a disadvantaged

628
00:43:26,050 --> 00:43:29,469
neighborhood you could do actually really poorly if you come from a

629
00:43:29,469 --> 00:43:34,210
privileged neighborhood you do pretty much as well as a school this high

630
00:43:34,210 --> 00:43:39,289
performing in Spain. What's so interesting about the Finnish schools is that the impact

631
00:43:39,289 --> 00:43:44,550
of social background doesn't make so much of a difference. The line is also a little

632
00:43:44,550 --> 00:43:49,050
bit shorter. One has to admit that. The line in Finland is not as long as in France. That

633
00:43:49,050 --> 00:43:53,489
means society in Finland is somewhat more homogeneous than society in France and Spain.

634
00:43:53,889 --> 00:43:57,710
That's something we have to take into account. But you can see it reflected here. But the

635
00:43:57,710 --> 00:44:01,769
point about Finland is that in Finland it doesn't matter so much whether your student

636
00:44:01,769 --> 00:44:05,469
comes from a wealthy or disadvantaged family back home,

637
00:44:05,809 --> 00:44:07,869
schools come out more or less the same.

638
00:44:09,010 --> 00:44:11,829
And Spain, as you could see before, is somewhere in the middle.

639
00:44:12,909 --> 00:44:15,050
But now what you can do is you can actually see

640
00:44:15,050 --> 00:44:17,469
how does my school compare against schools in Finland.

641
00:44:17,590 --> 00:44:19,429
And actually what you can see here is that

642
00:44:19,429 --> 00:44:22,869
actually that school in Spain that did so well

643
00:44:22,869 --> 00:44:25,070
also would have done well in Finland.

644
00:44:26,610 --> 00:44:28,369
Sort of a world-class school.

645
00:44:28,750 --> 00:44:30,889
It also would have done really well in France.

646
00:44:31,769 --> 00:44:34,210
It is actually quite competitive internationally.

647
00:44:35,110 --> 00:44:37,409
Had I taken a school at the lower spectrum,

648
00:44:38,150 --> 00:44:40,130
well, you know, the answer might have been different,

649
00:44:40,469 --> 00:44:41,929
and you might not have seen the success

650
00:44:41,929 --> 00:44:44,730
that you actually do see here on that chart.

651
00:44:45,030 --> 00:44:47,389
You can look at this by level of proficiency.

652
00:44:47,610 --> 00:44:49,170
You know, one of the things that we always do

653
00:44:49,170 --> 00:44:51,190
is we don't only give students a score.

654
00:44:51,869 --> 00:44:53,949
We assign them levels of proficiency.

655
00:44:53,949 --> 00:44:56,110
We want to actually explain what is it

656
00:44:56,110 --> 00:44:57,989
that those students can and cannot do.

657
00:44:58,449 --> 00:44:59,150
It's very important.

658
00:44:59,230 --> 00:45:00,130
It's very important concept.

659
00:45:00,130 --> 00:45:14,130
You don't only get a score of 510 or 470, but actually PISA can tell you this student is capable to read but has not developed the reading skills for learning.

660
00:45:14,130 --> 00:45:20,130
You can say those kinds of things depending on the level and you can see how basically your students perform.

661
00:45:20,130 --> 00:45:24,130
You can also see how they perform on different task types.

662
00:45:24,130 --> 00:45:27,489
For example, again, you know, I mentioned this at the beginning.

663
00:45:28,070 --> 00:45:31,010
We look at what students know of science.

664
00:45:31,530 --> 00:45:34,489
Do they know something in biology, chemistry, and physics?

665
00:45:36,030 --> 00:45:37,389
Is that the strength of your school,

666
00:45:37,530 --> 00:45:40,730
or is the strength of your school that students can think like a scientist?

667
00:45:41,570 --> 00:45:43,610
We can see both of those things.

668
00:45:43,889 --> 00:45:46,329
You can look at the share of top-performing students.

669
00:45:46,750 --> 00:45:50,110
Here you can see, for example, your school, that was the high-performing school,

670
00:45:50,489 --> 00:45:53,570
has about 5% of top-performing students,

671
00:45:53,570 --> 00:45:59,690
students who reach level 5 and 6, and actually, you know, you might think, oh, 5% is too low,

672
00:45:59,789 --> 00:46:04,789
but actually, I encourage you, you know, try out some of the PISA tasks yourself, level

673
00:46:04,789 --> 00:46:05,570
5 and 6.

674
00:46:06,289 --> 00:46:08,130
You'll find them really challenging.

675
00:46:08,809 --> 00:46:13,289
Students have to demonstrate some very complex mathematical or scientific thinking skills

676
00:46:13,289 --> 00:46:16,809
to get those tasks right, and you get basically 5 and 100.

677
00:46:17,070 --> 00:46:19,769
If you were in Finland, it would be not much more.

678
00:46:19,869 --> 00:46:20,889
It would be about 6.

679
00:46:20,889 --> 00:46:22,690
Now, Germany, also 6.

680
00:46:22,690 --> 00:46:24,190
Portugal, 3%.

681
00:46:24,190 --> 00:46:25,949
So you can actually compare yourself

682
00:46:25,949 --> 00:46:28,329
with whatever country you are interested in

683
00:46:28,329 --> 00:46:31,389
at the top and the bottom end of the performance distribution,

684
00:46:31,650 --> 00:46:33,349
which is also important, beyond the mean score.

685
00:46:34,349 --> 00:46:36,429
The last point I really want to make

686
00:46:36,429 --> 00:46:38,489
is actually that things can change.

687
00:46:40,309 --> 00:46:42,469
Here's a school in 2013.

688
00:46:42,789 --> 00:46:46,190
It was a wealthy school and a high-performing school

689
00:46:46,190 --> 00:46:47,690
in the case of Spain.

690
00:46:48,530 --> 00:46:52,130
And then you can see when we tested that school again in 2015,

691
00:46:52,690 --> 00:46:58,849
It had become a little bit more disadvantaged, but it had become a lot better.

692
00:46:58,849 --> 00:47:02,570
So actually we can see that some schools are moving very, very fast.

693
00:47:02,570 --> 00:47:10,969
Here's another one, 2013, and you can see actually in 2015 that school moved upwards.

694
00:47:10,969 --> 00:47:13,409
So it's not a static picture.

695
00:47:13,409 --> 00:47:17,610
Countries change, regions change, schools change in the learning outcomes.

696
00:47:17,610 --> 00:47:23,469
You can see yourself not only in a static picture, but increasingly in a kind of dynamic

697
00:47:23,469 --> 00:47:24,469
way.

698
00:47:24,469 --> 00:47:28,469
So in summary, you can benchmark your schools internationally.

699
00:47:28,469 --> 00:47:34,929
You can use PISA for schools for peer learning opportunities, to study practice, to drive

700
00:47:34,929 --> 00:47:38,789
practice, to explore new ideas, to bring people together.

701
00:47:38,789 --> 00:47:42,409
By the way, one of the things that we want to do either at the end of this year or the

702
00:47:42,409 --> 00:47:47,329
beginning of next year to bring actually schools together in Paris at the OECD

703
00:47:47,329 --> 00:47:52,130
internationally to look at this and we are also sort of creating a website

704
00:47:52,130 --> 00:47:58,429
where schools can connect that's all I wanted to share with you I hope we have

705
00:47:58,429 --> 00:48:04,369
stood some time for questions and I look forward to your question thank you very

706
00:48:04,369 --> 00:48:06,369
Thank you very much.

707
00:48:59,750 --> 00:49:08,000
I think it's a good question.

708
00:49:08,000 --> 00:49:15,139
I think what you can see is the share of students that do well and the share of students that

709
00:49:15,139 --> 00:49:17,000
don't do so well.

710
00:49:17,000 --> 00:49:23,639
You can also see to what extent student performance depends on the social background in your school.

711
00:49:23,639 --> 00:49:29,260
Is your school more or less successful than other schools in moderating social inequality?

712
00:49:29,260 --> 00:49:32,639
It's true that we are not looking at individual student results.

713
00:49:32,639 --> 00:49:35,840
It's not something the PISA for Schools test does.

714
00:49:35,840 --> 00:49:40,380
But it tells us something about the distribution of schools, of students.

715
00:49:40,380 --> 00:49:43,940
Sometimes we believe, well, you know, it's only natural that, you know, some students

716
00:49:43,940 --> 00:49:45,500
do well and others do not so well.

717
00:49:45,500 --> 00:49:48,599
But actually you can see that this varies across schools.

718
00:49:48,599 --> 00:49:54,920
Some schools are quite successful in moving all students along and others not so successful.

719
00:49:54,920 --> 00:49:59,400
And sometimes we can actually relate this to policy and practice.

720
00:49:59,400 --> 00:50:04,019
You know, Minister van Grieken mentioned the example of grade repetition.

721
00:50:04,019 --> 00:50:08,119
One device that students often use is, you know, if schools often use it, if a student

722
00:50:08,119 --> 00:50:11,679
doesn't do well, we'll let them do the same thing another year.

723
00:50:11,679 --> 00:50:18,039
Actually, if you look at the PISA results, it doesn't really help students to improve.

724
00:50:18,039 --> 00:50:22,820
It's something that doesn't help students to improve.

725
00:50:22,820 --> 00:50:23,820
It's very expensive.

726
00:50:23,820 --> 00:50:25,960
often stigmatizing for the student.

727
00:50:26,360 --> 00:50:28,820
If you look at what do school systems do

728
00:50:28,820 --> 00:50:31,659
that are very good in keeping all students moving forward,

729
00:50:32,059 --> 00:50:34,940
they are good in early diagnostics.

730
00:50:35,260 --> 00:50:36,519
Teachers spend a lot of time

731
00:50:36,519 --> 00:50:38,420
to actually find struggling students

732
00:50:38,420 --> 00:50:39,840
early on in their trajectory.

733
00:50:40,780 --> 00:50:43,139
They have a lot of time and resources to intervene,

734
00:50:43,260 --> 00:50:44,860
to help struggling students to succeed.

735
00:50:45,019 --> 00:50:46,679
So sometimes it's not a surprise

736
00:50:46,679 --> 00:50:48,800
that the distribution is narrower in one school

737
00:50:48,800 --> 00:50:49,880
than in another school,

738
00:50:49,980 --> 00:50:51,760
in one country than in another country.

739
00:50:51,760 --> 00:50:57,940
But there's a lot of attention being actually paid by PISA for schools to the distribution,

740
00:50:58,119 --> 00:51:03,199
even if deliberately we choose not to look at individual student results,

741
00:51:03,280 --> 00:51:05,920
because we would be worried that we have lists of students

742
00:51:05,920 --> 00:51:08,280
and it becomes a kind of different nature of the instrument.

743
00:51:14,699 --> 00:51:15,320
Hello, good morning.

744
00:51:15,739 --> 00:51:19,800
Being important as is the quality of learning for productivity,

745
00:51:19,800 --> 00:51:29,480
In the reports, the methodologies are made visible, which is the great debate that exists almost globally,

746
00:51:29,900 --> 00:51:33,440
if you see the methodologies, and that would be to imitate or export.

747
00:51:34,000 --> 00:51:41,929
It's a very interesting question. Of course, that's something that we study all the time in PISA.

748
00:51:42,849 --> 00:51:48,349
What are the kind of teaching and learning strategies that are relating to better learning outcomes?

749
00:51:48,349 --> 00:51:51,530
And indeed, there are some patterns, some very important patterns.

750
00:51:51,530 --> 00:51:56,130
For example, what we can see is that where there is a greater culture of teacher collaboration

751
00:51:56,130 --> 00:52:01,909
in a school, generally outcomes tend to be better, sort of a combination of professional

752
00:52:01,909 --> 00:52:06,789
autonomy and the collaborative in culture is a predictor of better learning outcomes

753
00:52:06,789 --> 00:52:07,789
in school.

754
00:52:07,789 --> 00:52:12,570
We've also looked at teaching strategy, and here some of our results were a little bit

755
00:52:12,570 --> 00:52:15,530
surprising for some.

756
00:52:15,530 --> 00:52:20,449
Many people say, well, if you want to teach science well, you should look at inquiry-based

757
00:52:20,449 --> 00:52:24,530
approaches to science, where students have a lot of hands-on experience and so on.

758
00:52:24,530 --> 00:52:31,449
You look at the PISA results, actually, those practices often did not relate to better outcomes.

759
00:52:31,449 --> 00:52:36,829
Maybe not because they're not done so well, maybe because the practice is not a good idea.

760
00:52:36,829 --> 00:52:42,010
Whereas some of the quite traditional teaching methods, teaching-directed approaches, actually

761
00:52:42,010 --> 00:52:43,690
were quite predictive for learning outcomes.

762
00:52:43,690 --> 00:52:48,889
So across countries, you can actually see some really interesting patterns that predict

763
00:52:48,889 --> 00:52:51,110
the quality of learning.

764
00:52:51,110 --> 00:52:57,909
You can also look at what effort do countries make to attract people into the teaching profession?

765
00:52:57,909 --> 00:53:01,889
What choices do they make between a better teacher and a smaller class?

766
00:53:01,889 --> 00:53:06,469
All of those kinds of things are quite apparent from the PISA study.

767
00:53:06,469 --> 00:53:11,090
How far you can extrapolate from that and apply to an individual school, that's a sort

768
00:53:11,090 --> 00:53:14,510
of question that every school, every country, every region needs to answer.

769
00:53:14,510 --> 00:53:19,530
This is just the broad picture that it gives, but PISA provides a lot of ideas that actually

770
00:53:19,530 --> 00:53:23,989
explain differences in the quality of the instructional environment.

771
00:53:23,989 --> 00:53:28,730
We have also looked at things, you know, one of the things that I found particularly interesting,

772
00:53:28,730 --> 00:53:35,389
we asked teachers themselves what kind of methods of teaching do they typically deploy.

773
00:53:35,389 --> 00:53:41,510
Do I use a more constructivist approach to teaching or a more kind of instrumental one?

774
00:53:41,510 --> 00:53:46,190
And then we can look actually how students learn, and we have asked the students the

775
00:53:46,190 --> 00:53:52,409
same question, you know, is in your instruction, in your lesson, is memorization dominant or

776
00:53:52,409 --> 00:53:56,730
elaboration strategy, deep learning strategy, and things like this prevalent.

777
00:53:56,730 --> 00:54:00,929
So we know from the teacher what they believe they are doing, and we know from the student

778
00:54:00,929 --> 00:54:06,829
what they are experiencing in the classroom, and we can put the two together.

779
00:54:06,829 --> 00:54:09,550
Sometimes they match perfectly in some countries.

780
00:54:09,550 --> 00:54:15,070
In other countries, they are totally different, and so you can actually evaluate the picture

781
00:54:15,070 --> 00:54:19,630
of instructional climate from many different lenses and perspectives.

782
00:54:19,630 --> 00:54:22,090
It doesn't always give you clear-cut answers.

783
00:54:22,090 --> 00:54:25,849
It only gives you sort of, it's a mosaic, you have to put stones in and it becomes more

784
00:54:25,849 --> 00:54:31,289
and more colorful over time, but it gives a lot of ideas about good instructional practice

785
00:54:31,289 --> 00:54:32,289
in countries.

786
00:54:32,289 --> 00:54:39,940
Buenos días.

787
00:54:41,019 --> 00:54:43,300
¿No existe la posibilidad en una prueba como esta

788
00:54:43,300 --> 00:54:45,079
que los centros, los países,

789
00:54:45,719 --> 00:54:47,840
preparemos la prueba para obtener mejores resultados

790
00:54:47,840 --> 00:54:51,800
en lugar de mejorar simplemente la educación de nuestros alumnos?

791
00:54:51,980 --> 00:54:52,260
Gracias.

792
00:54:58,480 --> 00:55:00,280
I think I must disappoint you.

793
00:55:00,860 --> 00:55:03,579
You know, if we had tested knowledge,

794
00:55:04,320 --> 00:55:05,519
you can prepare your students.

795
00:55:06,179 --> 00:55:06,719
That's easy.

796
00:55:08,059 --> 00:55:10,719
Testing competency is very hard to prepare your students.

797
00:55:10,719 --> 00:55:14,679
You know, PISA assesses, or the idea of this test is to test thinking skills.

798
00:55:15,340 --> 00:55:16,679
Can you think like a scientist?

799
00:55:17,619 --> 00:55:21,219
The best way you can prepare your students is to teach them well,

800
00:55:21,380 --> 00:55:23,519
is to teach them to think like a scientist.

801
00:55:23,920 --> 00:55:26,360
But there's no short-term strategy, unfortunately.

802
00:55:27,000 --> 00:55:31,619
There are no short-term wins, you know, teaching particular things in a particular lesson.

803
00:55:32,440 --> 00:55:33,699
Also, that's the one reason.

804
00:55:33,840 --> 00:55:35,480
PISA has a focus on competency.

805
00:55:35,480 --> 00:55:39,820
It doesn't lend the assessment to be what we call instructionally sensitive.

806
00:55:39,820 --> 00:55:42,320
to short-term things.

807
00:55:42,679 --> 00:55:43,579
The other part is

808
00:55:43,579 --> 00:55:46,139
we assess the cumulative yield of learning

809
00:55:46,139 --> 00:55:47,320
at PISA very intentionally.

810
00:55:47,599 --> 00:55:50,340
We do not test 10th grade knowledge.

811
00:55:51,380 --> 00:55:52,760
We are testing basically

812
00:55:52,760 --> 00:55:54,420
the cumulative yield

813
00:55:54,420 --> 00:55:55,420
from primary education

814
00:55:55,420 --> 00:55:56,699
up to 10th grade.

815
00:55:57,460 --> 00:55:59,559
So some of the PISA tasks,

816
00:56:00,119 --> 00:56:01,179
when you look at the mathematics,

817
00:56:01,320 --> 00:56:02,400
it's actually quite simple,

818
00:56:02,500 --> 00:56:04,199
but it's embedded in quite complex,

819
00:56:04,519 --> 00:56:05,579
demanding situations.

820
00:56:06,579 --> 00:56:08,360
So again, you cannot just,

821
00:56:08,599 --> 00:56:09,539
if you didn't do well

822
00:56:09,539 --> 00:56:11,400
between grade one and grade nine,

823
00:56:11,480 --> 00:56:13,980
and now suddenly you have a superb classroom in grade 10,

824
00:56:14,340 --> 00:56:16,199
you still will not get a good PISA score

825
00:56:16,199 --> 00:56:18,579
because it looks at the cumulative yield of education.

826
00:56:18,579 --> 00:56:23,480
So it's not a very instructionally sensitive test,

827
00:56:23,579 --> 00:56:26,239
so you can't do very much to prepare your students

828
00:56:26,239 --> 00:56:26,960
in the short term.

829
00:56:27,519 --> 00:56:29,039
We actually looked at...

830
00:56:29,039 --> 00:56:30,800
We did this quite intentionally like this

831
00:56:30,800 --> 00:56:33,199
because if it would be sort of a very specific,

832
00:56:33,639 --> 00:56:35,460
narrow test on 10th-grade knowledge,

833
00:56:35,460 --> 00:56:41,380
then, you know, it would have been too easy to do well on the PISA test

834
00:56:41,380 --> 00:56:46,960
and we wouldn't really reflect the quality of the education over a sufficiently long period of time.

835
00:57:20,719 --> 00:57:46,000
I would think more that among the remaining 15%,

836
00:57:46,000 --> 00:57:50,159
you'll find things like character qualities of the teacher,

837
00:57:50,719 --> 00:57:52,840
or things that are really hard to capture.

838
00:57:53,420 --> 00:57:55,579
You know, honestly, of course, you know,

839
00:57:55,639 --> 00:57:58,360
people talk about intelligence and those kinds of factors,

840
00:57:58,519 --> 00:58:01,699
but I, you know, become almost skeptical

841
00:58:01,699 --> 00:58:03,980
because I wouldn't want to think that, you know,

842
00:58:04,019 --> 00:58:05,880
Finnish students are inherently more intelligent

843
00:58:05,880 --> 00:58:08,679
than, you know, American students or a Spanish student.

844
00:58:08,840 --> 00:58:12,079
I do believe, actually, that a lot of the variability

845
00:58:12,079 --> 00:58:15,239
that we see comes down to what happens to those children

846
00:58:15,239 --> 00:58:18,000
in school or out of school in their experiences.

847
00:58:18,000 --> 00:58:25,159
Yeah, you know, intelligence may play a role, but it can't be a very, very significant role

848
00:58:25,159 --> 00:58:28,500
because the share of unexplained variation really is quite low.

849
00:58:28,900 --> 00:58:30,920
It may, of course, be entangled with social background.

850
00:58:31,059 --> 00:58:34,760
There may be lots of kind of interactions here that we need to look at carefully.

851
00:58:34,940 --> 00:58:40,860
But overall, I think most of the things that we can measure are ones that we can pretty well predict.

852
00:58:40,860 --> 00:58:48,280
And my hunch, really, that the remaining 15% will have to do with something that is very difficult to make tangible,

853
00:58:48,519 --> 00:58:55,119
like character qualities of teachers, enthusiasm, the kind of things that are not captured in formal training

854
00:58:55,119 --> 00:58:58,019
and qualifications of teachers as we measure them in the PISA.

855
00:58:59,320 --> 00:59:05,019
One of the things that we are going to do, and I'm very pleased that the region of Madrid is joining this as well,

856
00:59:05,019 --> 00:59:10,539
us actually to do a video study of teaching practices which will allow us

857
00:59:10,539 --> 00:59:14,079
to actually observe more closely what actually happens in the classroom and to

858
00:59:14,079 --> 00:59:18,159
derive you know policy lessons from that and I think maybe we get at the

859
00:59:18,159 --> 00:59:22,900
remaining 15% with those kinds of things but you know I don't want to dismiss the

860
00:59:22,900 --> 00:59:27,219
issue of intelligence but my general experience is that we attribute far too

861
00:59:27,219 --> 00:59:33,579
much importance to this like for social background whereas it from the results

862
00:59:33,579 --> 00:59:47,619
see clearly that it can't be such a big part of the story.

863
00:59:47,619 --> 00:59:54,719
Okay, you made a quick reference to the value of professional learning communities

864
00:59:54,719 --> 01:00:03,760
and how at the very heart of school transformation and education systems will be teacher collaborations.

865
01:00:03,760 --> 01:00:09,539
So there were some quick references and I was very impressed by some of them.

866
01:00:09,539 --> 01:00:13,900
I'm a firm believer in those if we really want to make a difference.

867
01:00:13,900 --> 01:00:22,000
So based on your knowledge of all the school systems, what would be two recommendations

868
01:00:22,000 --> 01:00:29,539
for our schools in Spain to enhance teacher collaboration in a way that makes a difference

869
01:00:29,539 --> 01:00:33,639
to our schools and our results for that matter?

870
01:00:33,639 --> 01:00:34,639
Thank you.

871
01:00:34,639 --> 01:00:35,639
Yeah.

872
01:00:35,639 --> 01:00:38,940
You know, I think there are things that you can do in a school, and there are things that

873
01:00:38,940 --> 01:00:48,480
you can do in a system. I start with an example from the system, like the city of Madrid.

874
01:00:48,480 --> 01:00:54,320
The city of Shanghai does something. For example, they have a digital platform where teachers

875
01:00:54,320 --> 01:01:00,880
can upload their lesson plans. If you have a great idea, a great project, an interesting

876
01:01:00,880 --> 01:01:05,239
lesson, a video or a taping, you can upload it, other teachers can see it. But the trick

877
01:01:05,239 --> 01:01:11,059
is a different one. The more other teachers in the province download your lessons, comment

878
01:01:11,059 --> 01:01:16,699
on your lessons, criticize your lessons, improve your lesson, use your lesson, the more popular

879
01:01:16,699 --> 01:01:21,599
you become in the system. And they have a reputational metric associated with this,

880
01:01:21,599 --> 01:01:26,420
you know, like we do on Facebook and on eBay. And at the end of the school year, your principal

881
01:01:26,420 --> 01:01:30,139
will not only ask you, you know, how well did you teach your own students, but what

882
01:01:30,139 --> 01:01:35,079
contribution did you make to the profession? What do other teachers think about your model

883
01:01:35,079 --> 01:01:38,119
And so that's sort of a lever you can deploy at the system level.

884
01:01:38,219 --> 01:01:40,480
It can be very effective to create a community of teachers.

885
01:01:41,460 --> 01:01:46,219
At the school level, professional learning communities are a very powerful tool as well.

886
01:01:46,440 --> 01:01:47,460
I'll give you one example.

887
01:01:48,800 --> 01:01:54,340
Our data show that it's very rare in Spain for teachers to observe other teachers' classrooms.

888
01:01:55,059 --> 01:01:58,519
One in ten teachers says that they've done it once in a year.

889
01:01:58,739 --> 01:01:59,199
It's nothing.

890
01:02:00,119 --> 01:02:03,860
If you go to many of the high-performing education systems,

891
01:02:03,920 --> 01:02:07,219
it will be like 50% or 60% or 70% of teachers.

892
01:02:07,940 --> 01:02:11,900
And that's a very powerful way to reflect on your own teaching practice

893
01:02:11,900 --> 01:02:14,619
by seeing how the similar lesson is being taught by another teacher.

894
01:02:16,039 --> 01:02:18,599
Easy to do, not expensive,

895
01:02:18,960 --> 01:02:21,500
and very powerful in shaping instructional practice.

896
01:02:21,739 --> 01:02:23,920
And it's important to organize this with peers.

897
01:02:24,099 --> 01:02:27,340
You know, it doesn't generally work well when it's done from the outside,

898
01:02:27,340 --> 01:02:29,179
You know, sending an inspector in the classroom.

899
01:02:29,659 --> 01:02:32,659
It's really something that schools can do better internally.

900
01:02:34,099 --> 01:02:37,480
And so I think classroom observation.

901
01:02:38,219 --> 01:02:42,119
The third element that we often find is appraisal and feedback.

902
01:02:43,079 --> 01:02:47,139
You know, one of the data points, and I'm sure the school is an exception to this,

903
01:02:47,219 --> 01:02:52,820
but one of the data points that surprised me when I looked at the PISA data and the TALIS data,

904
01:02:52,820 --> 01:02:58,059
that only about 25% of the teachers say

905
01:02:58,059 --> 01:03:00,780
their principal has ever talked to them

906
01:03:00,780 --> 01:03:02,400
about their pedagogical practice.

907
01:03:03,139 --> 01:03:05,340
So teachers are very much in isolation in the classroom,

908
01:03:05,539 --> 01:03:06,820
and they complain about this.

909
01:03:06,960 --> 01:03:08,340
When you ask teachers, actually,

910
01:03:09,519 --> 01:03:13,179
my job is getting more difficult every day.

911
01:03:13,460 --> 01:03:15,860
I mean, this is the one thing that we can say about teaching,

912
01:03:16,300 --> 01:03:18,460
that it's getting tougher every day.

913
01:03:18,460 --> 01:03:21,639
You get more difficult kind of social issues to deal with.

914
01:03:21,639 --> 01:03:23,699
the curriculum becomes more demanding

915
01:03:23,699 --> 01:03:25,019
all of those things happen

916
01:03:25,019 --> 01:03:26,659
but you're left alone in the classroom

917
01:03:26,659 --> 01:03:29,059
with a training that happened 20 years ago

918
01:03:29,059 --> 01:03:30,880
that's when you did your initial training

919
01:03:30,880 --> 01:03:33,900
there's little investment and little appraisal

920
01:03:33,900 --> 01:03:34,760
little feedback

921
01:03:34,760 --> 01:03:38,719
it becomes even more surprising

922
01:03:38,719 --> 01:03:41,099
principals speak rarely with their teachers

923
01:03:41,099 --> 01:03:42,760
about professional practice in Spain

924
01:03:42,760 --> 01:03:44,699
but even the fellow teachers rarely

925
01:03:44,699 --> 01:03:46,539
there's very little interaction

926
01:03:46,539 --> 01:03:49,019
there's a lot of exchange and coordination

927
01:03:49,019 --> 01:03:50,320
in Spanish schools

928
01:03:50,320 --> 01:03:53,820
Also sharing instructional material resources, teachers work on that.

929
01:03:53,820 --> 01:03:58,079
But when you look at the deep professional collaboration that really makes a difference,

930
01:03:58,079 --> 01:04:03,320
classroom observations, appraisal feedback, sharing really sort of very little of that

931
01:04:03,320 --> 01:04:04,320
happens.

932
01:04:04,320 --> 01:04:07,980
And I think this is something that is a very powerful .

933
01:04:07,980 --> 01:04:14,079
It comes also back perhaps a little bit to the model, the philosophy that schools use

934
01:04:14,079 --> 01:04:16,280
to teach.

935
01:04:16,280 --> 01:04:21,320
If you look at the work model in, I mean, maybe this is a bit too stereotypical, but

936
01:04:21,320 --> 01:04:24,900
in Spain, you know, a teacher is someone who teaches in the classroom.

937
01:04:24,900 --> 01:04:28,019
That's a little bit how the work organization is defined.

938
01:04:28,019 --> 01:04:33,679
If you look to many of the high-performing schools and countries, teaching is just one

939
01:04:33,679 --> 01:04:38,739
of many responsibilities that teachers have, and a working day includes, you know, less

940
01:04:38,739 --> 01:04:46,119
teaching and a lot more other things, and I think that is a very important ingredient

941
01:04:46,119 --> 01:04:49,300
That's harder to change because it impacts on things like, you know,

942
01:04:49,360 --> 01:04:51,400
you have to trade your office classes or more teachers.

943
01:04:51,579 --> 01:04:52,920
And so that's more difficult to change.

944
01:04:53,019 --> 01:04:58,159
But this kind of war culture also creates a kind of spirit that teaches, you know.

945
01:04:58,400 --> 01:05:02,679
In Japan, for example, lesson study is a very important part of this.

946
01:05:04,019 --> 01:05:08,920
I was, you know, I had once a visit in a very poor area in China.

947
01:05:09,260 --> 01:05:12,420
And I was so impressed how they were dealing with very disadvantaged children.

948
01:05:12,420 --> 01:05:15,460
You know, this was the first generation of children that was educated there, really.

949
01:05:15,460 --> 01:05:16,719
The parents had no education.

950
01:05:17,539 --> 01:05:19,219
And then I asked the teacher, you know,

951
01:05:19,760 --> 01:05:22,480
how do you actually keep contact with those parents?

952
01:05:22,679 --> 01:05:25,820
You know, this is a totally different world in the school for them.

953
01:05:26,659 --> 01:05:30,000
And she said, oh, well, you know, I call every parent about twice per week

954
01:05:30,000 --> 01:05:32,619
to talk to them and to help them with parenting,

955
01:05:33,000 --> 01:05:34,840
with, you know, education, with all of this.

956
01:05:35,579 --> 01:05:36,780
And then I said to her, well, you know,

957
01:05:36,800 --> 01:05:39,119
that must be adding up to a lot of your time.

958
01:05:39,199 --> 01:05:41,059
It must be a huge burden on your shoulders.

959
01:05:41,960 --> 01:05:44,699
And she said, well, yeah, maybe,

960
01:05:44,699 --> 01:05:48,699
but actually I never thought about it like this you know if I would if these

961
01:05:48,699 --> 01:05:52,820
50 parents wouldn't help me I could do my job you know this is part you know I

962
01:05:52,820 --> 01:05:57,360
have to leverage their capacity to help me teaching 50 students in my classroom

963
01:05:57,360 --> 01:06:02,460
and so again it's part of the work organization and it's creating a much

964
01:06:02,460 --> 01:06:07,099
more collaborative climate between teachers and schools but so again there

965
01:06:07,099 --> 01:06:11,579
are some easy things you know creating platforms and networks for teachers easy

966
01:06:11,579 --> 01:06:15,900
to do system-wide there are some things that depend very much on the leadership

967
01:06:15,900 --> 01:06:20,400
within the school and the climate for collaboration and then there are things

968
01:06:20,400 --> 01:06:25,400
that are you know require changes in the work organization and the changes in the

969
01:06:25,400 --> 01:06:29,500
concept of what a teacher's role really is