Defusing and Decoding AI for Maximum Fulfillment: Dylan Wiliam
How do you safely define 'peak humanity' and what would you consider its main characteristics?
What ingredients make the education, reasoning, rationality, and creativity for an era of fulfilment for humanity; particularly when faced with dramatic and powerful tools such as AI?
Dylan Wiliam is one of the best known educationalists in the world. He’s dedicated his entire career to understanding how to ensure learner outcomes are achieved, especially through the study of formative assessment. Dylan is currently Emeritus Professor of Educational Assessment at the Institute of Education, University of London. He was previously Deputy Director of the Institute, and has held posts at King's College London and Princeton University. He earned his PhD from the University of London in 1993.
We highly recommend one of his recent and many publications about preparing for AI: https://edarxiv.org/372vr/
—
This Season is done in partnership with Salzburg Global Seminar. https://www.salzburgglobal.org/
Please check out our partner’s publication advocating for education transformation: https://www.diplomaticourier.com/issue/transformed-the-case-for-education-transformation
—
[TRANSCRIPT AUTO-GENERATED]
00:00:06:07 - 00:00:12:18
Louka Parry
Hello everyone and welcome to the Learning Future Podcast. Today we're speaking with Dylan Wiliam. It's fantastic to have you here. Dylan, thanks for joining us.
00:00:12:21 - 00:00:13:24
Dylan Wiliam
You're welcome.
00:00:14:01 - 00:00:27:02
Louka Parry
Look, you know, I've been following your work, like many of us in education for, uh, for many years now. One question that we begin with here is what's something that you're learning right now? What's kind of at the forefront of your mind in terms of your own development?
00:00:27:02 - 00:00:34:06
Dylan Wiliam
I think moments, my major learning is around trying to figure out the impact of artificial intelligence on education.
00:00:34:06 - 00:00:52:12
Dylan Wiliam
for a long time was very kind of dismissive of what it could do. And so I think that's what I'm trying to figure out now, is to what extent we can control the tendency of AI to confabullate to allucinate make up stuff and make a genuine score.
00:00:52:14 - 00:01:00:13
Dylan Wiliam
And so that's what I'm trying to get my head around now is just what could be the productive uses of AI in the next, say, 5 to 10 years.
00:01:00:15 - 00:01:23:24
Louka Parry
Interesting Dylan. What what do you think? I mean, as I'll work at the learning future and your own really know is how do how do we ensure that the outcomes are being met in these education systems? And perhaps more of a transformation agenda is the orientation that we have rather than the improvement of the old model. Is your view that we are now at this tipping point where generative AI and large language models will just like in some ways you are the assessment grew
00:01:24:01 - 00:01:29:17
Louka Parry
You know, clearly that's its challenge. Some of the traditional summative assessments. In any case.
00:01:29:19 - 00:01:46:21
Dylan Wiliam
I'm not sure they are. I'm not sure they are. So I think that generative AI will be very powerful in terms of lesson planning, in terms of objectives, you know, just making those kinds of things that teachers are often required to do. They'll be much easier.
00:01:46:23 - 00:01:48:01
Louka Parry
Yes.
00:01:48:03 - 00:01:54:01
Dylan Wiliam
I think that they will basically kill homework or assessed homework anyway.
00:01:54:07 - 00:01:54:20
Louka Parry
Interesting.
00:01:54:20 - 00:02:28:22
Dylan Wiliam
And the Economist says your homework is dead. Because way of knowing who did it. Yes. But this is actually quite simple. You set the students a chapter to read and then you actually have a class discussion. Your wisdom at that point of the chapter when they return to the classroom. So I think that the traditional model for homework where you grade what the students bring back with them on paper is dead, but what they bring back with them in terms of what's in their heads, I think that's going to be very powerful and at least very skeptical about assessment.
00:02:28:24 - 00:02:29:08
Louka Parry
Yes.
00:02:29:09 - 00:02:59:18
Dylan Wiliam
Is because we will certainly be able now to come up with some very sophisticated simulations, just put students in an environment. But there's a fundamental limitation in the technology. So people love authentic assessments. Isn't it great to get students, students doing things that are real? How students do? How well students do on those tasks depends on whether they were lucky in the particular task they were assigned.
00:02:59:20 - 00:03:27:05
Dylan Wiliam
And so this is what's called a task performance interaction. So this goes all the way through from primary school, all the way through to medical training and medical license. So if you give people an authentic task, whether they were lucky in the particular task, they were given matters more than how good they are, how hard the task was, or who did the marking combined.
00:03:27:07 - 00:03:28:06
Louka Parry
Interesting.
00:03:28:06 - 00:04:09:15
Dylan Wiliam
So there is a fundamental limitation of the reliability of these assessments. You know, we're promised that these AI bolts will diagnose students strengths and weaknesses, but they're not going to be very, very useful because we've estimated that you probably for, say, secondary school mathematics, you would need to know 300 different things about a student to optimize instruction. There are 300 little skills that they need that you need to know about in order to design the most effective instruction and to get accurate data on each of those 300, you probably need probably 30 items.
00:04:09:17 - 00:04:36:23
Dylan Wiliam
I see you're talking about 10,000 assessments just to be able to do good diagnostic mapping of students and competencies. And I don't think that's going to happen. So my my strong belief is that the assessment part is not going to change that much. The good news is we are going to be able to get assessments being scored by machines quite accurately.
00:04:37:02 - 00:05:02:23
Dylan Wiliam
I don't know if you know this, but computers are already as accurate as human beings are at marking essays. Yeah, that sounds impressive. Do you realize how bad humans are? So, you know, I think we all going to be able to have much more authentic assessment, but we need a lot more of it before we actually get a robust reading of what that student is capable of because we need to test them in different contexts.
00:05:03:00 - 00:05:09:00
Dylan Wiliam
So so my view is it's going to make much more of a difference to the teaching side than it will to the assessment side.
00:05:09:02 - 00:05:32:14
Louka Parry
Fascinating. So, you know, you've been talking, I guess, a bit about I mean, we are in this hype cycle always dealing with the new technology that's come through. And so, you know, right at the top of the curve, you know, have to go to the trough of disillusionment just yet. But I do wonder about some of some of your your thinking and the team's thinking about I guess I would like to talk about the positives around the teacher workload side in particular.
00:05:32:14 - 00:05:52:03
Louka Parry
You know, this teacher crisis happening elsewhere. But before we get to that, I just wondered about the pitfalls specifically that you see with these these types of technologies. I mean, we know the Center for Humane Technology, for example, they spoke about the social dilemma. They talk about the A.I. dilemma. Do you see some kind of more macro impact on kind of the cohort of young people of even cognition itself?
00:05:52:05 - 00:06:08:02
Louka Parry
Now that's been kind of everything is being augmented in some ways or, you know, automated and, you know, every consulting, large consulting organization, BCG, McKinsey, everyone's talking about hundreds of millions of jobs, really transforming. What's your kind of macro view before we go into kind of the school year?
00:06:08:04 - 00:06:23:17
Dylan Wiliam
I think you have to think carefully about what those people are actually saying. So the latest estimates from a McKinsey report, I think, was that something like 30% of the jobs that are currently being done aren't going to be affected at all by the AI.
00:06:26:21 - 00:06:50:23
Dylan Wiliam
A small number of jobs are going to be completely eradicated. I would say something like 35 to 40% of the work currently being done is going to be done by AI. Right. But it also generates extraordinary opportunities for the people to do other things. And so for me, the story of technology is not that when you actually have machines doing humans work, that humans are left to do nothing.
00:06:51:03 - 00:06:59:05
Dylan Wiliam
They left to do other things that are more interesting and more exciting. So I don't see this lump of labor fallacy that people often talk about.
00:06:59:07 - 00:07:00:07
Louka Parry
Yes.
00:07:00:09 - 00:07:18:03
Dylan Wiliam
There's only a certain number of jobs to go around. I think we'll find a few things for those people to do. And because the machines will actually allow us to be more productive, we'll be wealthier and we can pay for more of those kinds of things that currently are only enjoyed by an elite. Yes, that is that is a 2000 year story.
00:07:18:06 - 00:07:31:21
Dylan Wiliam
Yes. Yeah. What happened? It was around he had somebody getting up at 3:00 in the morning to light a fire in every room that he would be walking into during the day. He enjoyed a very low standard of living.
00:07:31:23 - 00:07:33:03
Louka Parry
Interesting.
00:07:33:05 - 00:07:48:15
Dylan Wiliam
And very, very few other people did. You know? Now, most of us can enjoy that kind of standard of living. So what I think technology is doing is giving to most people what elites have enjoyed for, if not forever, for a certainly for a very long time.
00:07:48:17 - 00:08:08:13
Louka Parry
Oh, fantastic. Yeah, I what I'm curious about in that, Dylan, is how you know, I think I think there's been the promise of technology being this great equalizer for a long time and clearly has you know, you look at Steven Pinker's work or others saying, you know, the world in particular is better off now, you know, in general than it has been previously.
00:08:08:13 - 00:08:37:23
Louka Parry
Yeah. Notwithstanding, you know, many crises, etcetera, like climate. But I wonder about like what's the missing piece here? Because if technology had done that, you know, even Jules Verne was talking about all this, you know, all the kind of a past futurists have said eventually we'll only need to work 10 hours a week because, you know, the increases in production capacity and the industrial revolutions, plural, means that, you know, we'll actually be able to do what education should be for which is, you know, how to spend one's life well, including leisure time, not just in terms of productive capacity.
00:08:37:23 - 00:09:11:19
Louka Parry
So, you know, for me that the piece is education. Clearly, it's not kind of a techno optimist. We let the tech lead. It seems like that's the tool that helps with the transformation needed at a societal or system level. When we're speaking about education systems in particular, what's your sense of the absolutely essential elements? Now that we have this technology that I agree with you, gives us this opportunity to in some ways completely rethink and reimagine kind of the workforce and what work might be for to what's higher order, more fulfilling, you know, more meaningful aspects.
00:09:11:19 - 00:09:26:08
Louka Parry
The Gallup polls always talk about the disengagement at work, you know, as being astronomical and apathy being so high and nihilism on that kind of point in. And it is a big picture question, but I think I really think this is kind of the nuts of it. And then we kind of talk about the way education systems are oriented.
00:09:26:10 - 00:09:27:19
Louka Parry
So what's your sense?
00:09:27:21 - 00:09:52:03
Dylan Wiliam
Well, well, first of all, you're Jules Verne later Caines Galbraith Yes. They were all talking about, you know, we will be able to just do 10 hours work a week. And they were right. We would be able we would be able to maintain the standard of living that the average person had in the 1930s. On working a week.
00:09:52:05 - 00:10:15:11
Dylan Wiliam
But what we've chosen to do is to work more and have more money to be able to do more stuff. Yes, So that tells me something about humans. So I think that a lot of people enjoy work, they like work, they like the community of it, but they also like the the engagement that work brings in work is dignifying workers meaningful lives.
00:10:15:13 - 00:10:42:21
Dylan Wiliam
So I'm very skeptical about, oh, look, the likelihood of people choosing not to work. Say there's universal basic income. Yes, no. Now said, you know, the Greeks, the ancient Greeks, the elite amongst the ancient Greeks enjoyed the fact that they didn't actually have to work and they could spend their time disputing and thinking and drawing geometrical figures in the sand or whatever, you know, enjoy the life of the mind.
00:10:42:21 - 00:11:09:01
Dylan Wiliam
And we may get that. I just don't know. But I'm just I'm persuaded by this idea that given the fact that people could live much simpler alives and earn much less money, they chose not to do that. And therefore, I suspect that we will use the productivity to actually do more stuff, have more things. The other thing to remember, of course, is does that stuff become sustainable?
00:11:09:01 - 00:11:44:13
Dylan Wiliam
Because as economists like Mark and recently Erik Brynjolfsson of the show, we actually get more from less. So your iPhone replaces a reel to reel tape recorder, a movie camera. You know, it's incredible how the technology allows us to have a smaller impact on the planet. I'm hoping that this technology will help us with nuclear fusion, for example, and in which case global warming is going to be pretty much a non-issue in 5200 years time because we've actually cracked those issues.
00:11:44:13 - 00:12:05:04
Dylan Wiliam
We also have carbon carbon capture, and I think the technology will help us with all of that. So I'm quite bullish about the possibilities of it, but mean this is quite recent, which is why, you know, I collaborated with Aaron Hamilton and John Howard to I am concerned about some of the negatives and I just think we're not thinking about them seriously enough.
00:12:05:06 - 00:12:06:17
Louka Parry
Yes, that's the problem.
00:12:06:19 - 00:12:36:17
Dylan Wiliam
Yeah, it's the it's the precautionary principle. Yeah. To use the the metaphor that Nassim Nicholas Taleb uses, if you're told that the average depth of the river is three feet and you're five feet tall, you think you're okay. But if the average is three feet, but there's one bit which is 20 feet deep, then you're in trouble. So it's the kind of assumption that the world is knowable or predictable that I think is so dangerous.
00:12:36:17 - 00:12:43:11
Dylan Wiliam
That's why I think the precautionary with this will need serious addressing, even if we don't. I mean, I don't see the prospect of banning AI.
00:12:45:07 - 00:13:10:12
Dylan Wiliam
Because even if people in the West agreed to do that, there's no prospect of that happening in China or Russia. And so, you know, I think we have to kind of do what we can to to get people to agree to limit the use on a global level, as we did with nuclear weapons. Yes. And even that hasn't perfect because we've still got you, you know, rogue states using nuclear weapons, developing them at least.
00:13:10:14 - 00:13:34:21
Dylan Wiliam
Yes. So that's I think that that's the challenge is the is the it'll allow us to be much more productive and much more powerful. The question for me is what will allow us to do that? The day that the machines can't do. And so I think we used to think that creativity was building these machines, but actually they're pretty good at painting that point.
00:13:34:23 - 00:13:36:13
Dylan Wiliam
They're pretty good at composing.
00:13:36:15 - 00:13:37:24
Louka Parry
Yes.
00:13:38:01 - 00:14:13:20
Dylan Wiliam
But but what I don't yet see is any kind of generative artificial intelligence being able to improvise in a group with others listening, like a jazz group listening to what other people are doing and riffing off that and reacting to that. So, you know, I do see that, for example, in the music side. Yes, I hope that we continue to educate young people to play musical instruments so they can actually enjoy the social of playing, which is to play with other people, particularly in an improvisatory way.
00:14:13:22 - 00:14:45:00
Louka Parry
Fantastic. That's so brilliant for so many, many different buttons there. The precautionary principle, I think this move fast and break things mantra, that was Facebook's initial mantra is now being completely consumed by again this AI movement. I think it's just going to be so fascinating. I think your idea around well, actually the parallels with kind of the nuclear disarmament least the limitation of some of these technologies, which of course will be transformative and save lives and improve them hugely, but also might distract us from our own lives in some ways.
00:14:45:00 - 00:15:16:05
Louka Parry
You know, I'm very curious about in fact the duration between consumption and creation and so your beautiful metaphor and, you know, literal example of music, you know, this idea that the beautiful thing about jazz, it is emergent creation. Yeah, it's constant feedback. Talk about form. It is constantly a tuning into group flow dynamics and, you know, even some of the beautiful work that we see around, you know, brainwaves actually also starting to synchronize in particular ways, which is also what happens in high performing teams and in lots of different contexts.
00:15:16:05 - 00:15:39:09
Louka Parry
So yeah, maybe more flow. I think, you know, really interesting that how we architect for flow in particular, you know that peak capacity a peak challenge as much as possible. And surely some of these tools will get us halfway there. Gosh so good. How then, Dylan, do we think about the kind of core elements for the education system of the future to your proms?
00:15:39:09 - 00:16:09:16
Louka Parry
And this is one I always reflect on is how how might we become fully human in this of this AI world, right? This increasingly in some ways disembodied world where our cognition is being extended by other tools in what are these key features of of, you know, beautiful learning organizations, schools that where people go to convene? Or do we all just end up in a kind of VR metaverse city type model?
00:16:09:16 - 00:16:11:19
Louka Parry
I mean, who knows? I mean.
00:16:11:21 - 00:16:24:02
Dylan Wiliam
Anybody who thinks they might have the answers is just deluding themselves. But I go back to the stuff that was done in the 1970s and eighties on distributed cognition.
00:16:24:06 - 00:16:24:21
Louka Parry
Yes.
00:16:24:23 - 00:16:52:03
Dylan Wiliam
So it's a very nice work. But like Gabrielle Solomon, who, you know, we talk about things like the the long multiplication algorithm, how do you multiply two numbers together? And the nice thing about that is it's what David Perkins calls Person Plus. You've got the you've got the brain, but that becomes more powerful when you give them a pencil and some paper and some ideas about how to really organize that calculation.
00:16:52:05 - 00:17:21:20
Dylan Wiliam
And I think the whole idea of not just physically distributed but also socially distributed copies, I think like I see a machine is helping with that. At the very basic level, I can see software monitoring what students are producing at that of their laptops. And if a teacher wants to organize the kids for heterogenous groups, you press a button, suggest some groupings for that.
00:17:21:22 - 00:17:43:24
Dylan Wiliam
If you want to organize more homogenous groups, you want to put create some students. We've got to really push each other at the top end. And I think that's what my concerns with mixed reality teaching is Often the very, very precocious students are intellectually isolated. Yes. So you might want to actually put those kids in a separate group and push them really hard.
00:17:44:01 - 00:17:58:10
Dylan Wiliam
And I certainly think that generative A.I. will be very helpful in helping. You know, you just come up with ideas to really stretch children. I think just learning really about product architecture.
00:17:58:12 - 00:17:58:19
Louka Parry
Yes.
00:17:58:20 - 00:18:30:17
Dylan Wiliam
You know, people give Chat GPT a prompt, They see anything. Okay. Yes. So. So, so. But what's really interesting is if you iterate, if you go back in a now, recently, I think within the last week or so, Chat GPT has made a completely available customization. So you could actually tell chat GPT and then the example that Open ai gave was that what are the important things about touch that people need to know about the moon?
00:18:30:20 - 00:18:34:05
Dylan Wiliam
Okay, that's the general robot, but you can tell it.
00:18:34:05 - 00:18:36:03
Dylan Wiliam
I'm a third grade teacher.
00:18:36:05 - 00:18:37:23
Louka Parry
Yes.
00:18:38:00 - 00:18:39:17
Dylan Wiliam
And so you could actually know
00:18:39:17 - 00:18:41:00
Louka Parry
Interesting.
00:18:41:02 - 00:18:56:23
Dylan Wiliam
Tell it what kinds of things you'd be interested in so you can short circuit that learning that you get from the prompt. And so I think that we could see lots of really interesting ways of generating activities. But I think we're going have to keep the humans in the loop.
00:18:57:00 - 00:18:57:19
Louka Parry
Yeah.
00:18:57:21 - 00:19:06:09
Dylan Wiliam
So in the same way that the best chess player in the world is no longer a human, but nor is a machine, it's teams of humans working with machines.
00:19:06:15 - 00:19:07:06
Louka Parry
Right?
00:19:07:08 - 00:19:25:14
Dylan Wiliam
I think that the best teachers are going to be people who are working with the technology to actually come up with ideas, but it's still the teacher who becomes the curator, if you like. Yes. Learning material and selects that. Yeah, I think that I think that task would work really well with these students, but I think we need to modify it for these students.
00:19:25:14 - 00:19:38:03
Dylan Wiliam
I think that is another very powerful ease in the technology provided the teacher is there as a curator rather than just delegating the issue of tasks to the machine.
00:19:38:03 - 00:19:58:21
Louka Parry
Hmm. Brilliant. Dylan. I remember one of the graphs actually in that paper that Aaron John in itself put together. You know, it showed, I think, the difference between a novice and novice educator with novice I this is an expert educator with, with expert I was talking 2 to 5% more productivity and that's not surprising. I think one thing that we do forget is this These AIs are trainable.
00:19:58:23 - 00:20:29:08
Louka Parry
They you know, there are these large language models and you can effectively spin up, you know, multiple AIs in one and have them in kind of some kind of virtual avatar, so to speak this idea of, you know, and this is this, of course, I'd love you to focus on, you know, the role of the teacher, the shifting in this landscape away from kind of the instructor the night what's very much just like this architect of a learning experience, a curator in some ways of, you know, rigorous, high quality content, but ultimately competency potentially as the goal of a lot of education systems.
00:20:29:10 - 00:20:45:13
Louka Parry
What's your reflection around the shifting role of the teacher, knowing that here in Australia and the UK, the US, in many places around the world, there is just a complete crisis in terms of recruitment and retention of these wonderful human beings that are trying really to support young people to thrive in this uncertain world.
00:20:45:15 - 00:21:04:24
Dylan Wiliam
Well, I think that the answer is going to have to be lower contact time. So I think in Singapore and Shanghai and Taiwan, Hong Kong, it's quite common to have classes of 40 and they do that in order to have 50% on contact time.
00:21:05:05 - 00:21:05:13
Louka Parry
Yeah.
00:21:05:17 - 00:21:26:12
Dylan Wiliam
So the middle school, secondary school teachers teaching 13 to 15 hours a week. I think we have to get that in Australia and the UK and the US because the only then is the teacher going to have the time to become competent enough with the eye moving at the speed it's moving just to, just to just keep up with that, to keep up.
00:21:26:13 - 00:21:27:01
Louka Parry
Great point.
00:21:27:07 - 00:21:49:08
Dylan Wiliam
And that means because I don't think the behaviour of young people in schools in the US and the UK in Australia is good enough. I think we are going to have to spend much more money on education. So I think a substantial proportion of the dividend from the increased productivity that AI brings is going to have to go to education.
00:21:49:14 - 00:22:25:02
Dylan Wiliam
I think we probably need to double education expenditure. In order to have existing staff student ratios, but 50% non-contact time for teachers. People always talk about the cost of education. They always point out that we haven't seen any increases in productivity in teaching. You say the budget keeps on going up with the results don't get any better, but I keep on pointing out that it takes the same amount of time to play a mozart string quartet today than it did when Mozart wrote it hundreds of years ago.
00:22:25:04 - 00:22:28:21
Dylan Wiliam
And what's more, that used to live four people.
00:22:30:03 - 00:22:59:08
Dylan Wiliam
Haven't been any productivity improvement in string quartets. Yes, that's why the cost of education keep on going up. It's what William Beaumont calls a cost disease. You can't pay. You can't index teacher salaries or position salaries on the price of goods. You have to index it on average earnings for the good. In most countries, which earnings increase faster than the price of goods.
00:22:59:10 - 00:23:28:22
Dylan Wiliam
Opera, drama, theater will always increase faster than the rates of price inflation because of this cost of disease, as Bamako called it. I think you should face up to that. Yes, we're going to have to have an open ended increasing commitment to educational expenditure to produce the kind of citizens that will really flourish in as well.
00:23:28:24 - 00:23:49:03
Louka Parry
Not so good. And and your contention then, you know, because of course, when I was studying with John Hattie at University of Melbourne this year, the conversation around class sizes is just still prolific, really. I mean, it's it's one instrument. It's an interesting because people say would put more money and reduce the class sizes, but you're effectively advocating for the opposite.
00:23:49:03 - 00:23:50:18
Louka Parry
You said increase them. No.
00:23:50:20 - 00:24:17:09
Dylan Wiliam
No, no, no, not first of all, I think John's wrong about class sizes because he mixes up two things. So the point is that class size reduction policies don't work. And he's right about that. But he's wrong to say the class size doesn't matter if the same teachers teaching a smaller class, they'll get better results. They'll be even better if they get support and professional development in taking advantage of the smaller class.
00:24:17:11 - 00:24:18:07
Louka Parry
Yes.
00:24:18:09 - 00:24:28:02
Dylan Wiliam
But what does is he looks at the studies that have been done and the studies that have been done are the cheapest studies. The ones we don't reduce class size very much.
00:24:28:02 - 00:24:28:11
Louka Parry
Yes.
00:24:28:11 - 00:24:52:02
Dylan Wiliam
And you don't get time to figure out how to teach differently. And so, you know, educational research will tell you what was not what might be. And so I think we have to be very cautious about generalizing from existing research, especially when existing research is so problematic. And of course, the real problem class size reduction is in most countries it leads to a diminution of teacher quality.
00:24:52:02 - 00:25:16:12
Dylan Wiliam
You need more teachers. If you look, there is class size and in most countries when you reduce class teachers, you're adding aren't as good as the teachers you've already got. You've got the class size effect fight, teacher quality effect, because the teacher quality effect is generally larger than the class size effect. The net effect of class size reduction programs is often, to make things worse. But it's only because you increase the quality of teachers.
00:25:16:12 - 00:25:20:06
Dylan Wiliam
It's not because class size reduction doesn't work.
00:25:20:08 - 00:25:41:03
Louka Parry
How do you understand this? This question of what works? Because one of the quotes that I cite you very routinely on is that what works is an incomplete question. It's what works for whom, where, under what conditions. You know, there's all these qualifiers that we forget, and yet for some rationale, we have kind of completely internalized this idea of just all we just do the evidence.
00:25:41:03 - 00:25:54:07
Louka Parry
We know what works. Just do that. Teacher. So how do you we create the right new ones conversation? I believe this is at the center of your work, you know, so that there is this sense of co agency with the educator. Yeah.
00:25:54:09 - 00:26:13:23
Dylan Wiliam
So I mean, that's why I don't know if you've heard me talk about this, but I've developed five questions, but I think we have to ask of any age good research, education, research, ok? Question one is, does it solve a problem we have? So we know that teacher subject knowledge correlates with student progress, but only up to a certain point.
00:26:14:00 - 00:26:36:09
Dylan Wiliam
If your teachers already got good subject knowledge, increasing it further won't improve student achievement. So the question what is doesn't solve a problem we have. The question too is how much extra achievement will we get measured in months of learning per year, not effect sizes. Yeah, because effect size is very too much across different contexts. Question three is how much will it cost?
00:26:36:11 - 00:26:38:07
Dylan Wiliam
Forget money. Teach your time.
00:26:38:09 - 00:26:39:13
Louka Parry
Yeah.
00:26:39:15 - 00:27:04:14
Dylan Wiliam
Opportunity cost is the single most important education reform. Every hour teachers spend on one thing is an hour. They don't have expensive deals. So how much would it cost in terms of teacher time? Question for is can we implement it here So close large reduction programs can work it expensive, but they can work if you have a plentiful supply of teachers waiting to be hired.
00:27:04:16 - 00:27:10:22
Dylan Wiliam
But a disastrous when teacher quality is weak and teacher recruitment is challenging.
00:27:10:24 - 00:27:12:02
Louka Parry
Hmm.
00:27:12:04 - 00:27:35:24
Dylan Wiliam
So the question four is can we? Question five is do we know what to do? So telling people if collaborative learning has an effect, size of four is useless unless you know what it means to a learning. And it turns out that most people who do collaborative learning don't do it in the way that the research suggests has a positive benefit.
00:27:36:01 - 00:27:47:13
Dylan Wiliam
So I think one of the problems is we've tried to synthesize or aggregate ideas. Yes. Rather than programs.
00:27:47:14 - 00:27:48:03
Louka Parry
Yes.
00:27:48:03 - 00:28:24:01
Dylan Wiliam
So I do want to know what whether teacher collective efficacy or collaborative or cooperative learning works. I want to know, does this particular approach, this particular program, produce achievement? And it comes out of something very simple. Most countries, certainly the rich countries, can probably find 25 hours every year for teacher professional development. So my question for every school leader is this: What use of those 25 hours a year will have the biggest benefits for students?
00:28:24:03 - 00:28:41:13
Dylan Wiliam
We can wish for teachers who are more skilled, more enthusiastic or whatever, but the teachers we've got, the teachers we've got. The question is, where are you going to go from here? So it comes down to what are you going to do? What use of those 25 hours a year is going to the biggest benefit for our students?
00:28:41:15 - 00:28:50:24
Dylan Wiliam
I mean, basically anything else is just a side note. It's really it's not going to have an impact.
00:28:51:01 - 00:29:19:00
Louka Parry
Yeah, I was curious that didn't how diplomatic you were going to be. I take side notes as a reflection. Look, that's absolutely brilliant. I mean, the idea of, you know, this is a practice science. This is an applied science. I think sometimes we confuse education and we kind of get theoretical and in some ways enjoy just, I guess, the consternation of disagreeing on these aspects as opposed to saying, you know, the reading wars, for example, is one such example, which I think is
00:29:19:02 - 00:29:23:07
Louka Parry
I'm always curious about, you know, people have such ideology and so the political.
00:29:24:15 - 00:29:51:04
Dylan Wiliam
And is thought to very, very unproductive. But I think we have to understand that, you know, when people compare education unfavorably with medicine or with physics, usually it's idealized view of physics and medicine. So Pickering's book on the mangle of Practice shows that these physicists were designing these experiments, but it was their technicians who are making the work of this Nobel Prize winning physicist moves to a different lab.
00:29:51:06 - 00:29:59:07
Dylan Wiliam
His experiments don't work anymore because actually it was the skill of the technician of the craft People who got these things to work.
00:29:59:09 - 00:29:59:13
Louka Parry
So good.
00:29:59:13 - 00:30:09:20
Dylan Wiliam
We know a lot of the science of medicine, but what we can't yet do is make sure the players who prescribe antibiotics actually complete the course.
00:30:09:22 - 00:30:12:01
Louka Parry
Yes, it's a great point.
00:30:12:03 - 00:30:26:05
Dylan Wiliam
And so whenever we compare education unfavorably with other sciences, we're always picking on the most scientific bits of those other practices. And I think all sciences have these problems. Once you get down into the detail.
00:30:26:07 - 00:30:26:14
Louka Parry
Yeah.
00:30:26:14 - 00:30:47:19
Dylan Wiliam
And I think that, you know, the world is complex and would be great if it were simple. It isn't. And that's why I think that the time that teachers hopefully will be given will allow them to become a critical consumers of educational research. They will actually engage with the research. They'll find out, but then they'll feel what they lose.
00:30:47:19 - 00:30:55:05
Dylan Wiliam
Those five questions that I mentioned. And if it doesn't make sense in our context, would it work with all parents, with all students.
00:30:56:14 - 00:31:26:15
Dylan Wiliam
Without employers? I think that is the way forward to give people time to just acknowledge the complexity of this. And so that's why, you know, for me, I would like to make teaching something where people have the time to think so that it becomes think that the smartest people in our country really aspire to do because they see it as exciting, intellectually challenging, as well as, being socially protected.
00:31:26:17 - 00:31:53:20
Louka Parry
Absolutely. Oh, that's so inspiring. I have to say, when you talk about the kind of the craftsmanship, the science, art craft, the kind of it's interesting convergence there with your view on the role of a teacher as a action researcher should they, you know, it's not it's not that they are I mean, because, you know, this is often thrown around all the time when an action research thing or doing a professional learning community approach
00:31:53:22 - 00:32:00:13
Louka Parry
What's your sense around the role of. A Because I don't know, is it researcher necessarily? It doesn't that doesn't feel like the right frame at all?
00:32:00:15 - 00:32:24:02
Dylan Wiliam
No. So basically my definition of research is inquiry that is designed to transcend the context of data collection. So the purpose of research is that you do research in one context and you do that in a way that allows you to draw conclusions about different contexts. And I don't think most teachers need to be doing that. I think they should be engaging in a process of principle inquiry.
00:32:24:04 - 00:32:55:20
Dylan Wiliam
They should be constantly striving to improve their practice by reference to the research, and some may wish to write that up, shared with other practitioners some of these videos or whatever. Some may wish to write it as an academic journal article. They should be encouraged to do that and give them the time to do that. But I don't think teachers should be actual researchers because you need a lot of time to develop the skills to be an effective researcher, to develop insights that transcend the context of data production.
00:32:55:22 - 00:33:21:03
Dylan Wiliam
And so for me, what I would like to see more productive partnerships between teachers and researchers and the for example, the the first formative assessment project that poor Black and I did with Chris Martin and Clara Lee, we actually recruited some teachers. We asked them who would which classes would you like to try this out with?
00:33:21:05 - 00:33:48:01
Dylan Wiliam
What measures of effectiveness could we use? What data you collected on these students anyway? What's the best comparison group? And so for each of these 24 teachers, we did 24 mini experiments. Yeah, where you are each teacher to bring their own data with them. And I sat down with them and we did the data analysis. Interestingly, then I then used a technique that Paul Holland, the statistician, called a poly experiment.
00:33:48:03 - 00:33:49:03
Louka Parry
Right?
00:33:49:05 - 00:33:55:05
Dylan Wiliam
I used effect sizes to find a way of combining these. I was basically inventing research methods as I went.
00:33:55:07 - 00:33:56:02
Louka Parry
Right.
00:33:56:04 - 00:34:24:16
Dylan Wiliam
At that paper has actually been the most cited paper in the journal in which it was published in 2004. Right. But it's interesting because a young researcher probably would be well advised to take that kind of risk of stepping away from the cookbook you published. You do the kind of research that's easy to get published. You know, I felt I was going out on a limb, but I didn't worry about that because I'd already become a full professor.
00:34:24:18 - 00:34:50:05
Dylan Wiliam
I didn't have anything to prove. And so, you know, I think that we need productive partnerships between researchers and teachers. But to do that, we have to change the incentive systems in universities so that teachers so that I can get credit for the kind of practice. So for the kinds of things the research studies that change classroom practice.
00:34:50:07 - 00:35:05:07
Louka Parry
I'm so glad you brought up incentives. I think, you know, we've seen a very wide ranging conversation so far. Economics said the incentives in education, when you talk about student outcomes or, you know, achievement, what do you mean specific? Because I'm really curious about it.
00:35:05:12 - 00:35:13:08
Dylan Wiliam
I know I've never defined it. I never defined it because I don't want to define it. Great. Because I want people to decide for themselves.
00:35:13:08 - 00:35:13:18
Louka Parry
Yeah.
00:35:13:19 - 00:35:32:06
Dylan Wiliam
And so I have very strong views about what I think education should be about. But I've also learned that if I try to push those views on people who work at a very constrained system, I'm going to have zero relevance because they're going to say I can't do that. So for the last 15 years, I've been asking teachers, What do you want to achieve?
00:35:32:08 - 00:35:33:04
Louka Parry
Right?
00:35:33:06 - 00:35:50:18
Dylan Wiliam
Let's say clear about that. I say, Well, here's my ideas about how you can do that more effectively. So, you know, I never tell teachers what to do because that just the constraints they have to negotiate in their day to day jobs are so complex that I've got time to learn about how to optimize those for them.
00:35:50:18 - 00:35:53:05
Dylan Wiliam
So I just say, you know, you decide what you want to teach.
00:35:54:01 - 00:35:58:05
Dylan Wiliam
Before you even begin, you need to tell me what you want to achieve, what outcomes you value.
00:35:58:07 - 00:35:58:16
Louka Parry
Yes.
00:35:58:16 - 00:36:00:08
Dylan Wiliam
And then I think I can help you.
00:36:00:10 - 00:36:23:02
Louka Parry
Fantastic talent. Gosh, that's brilliant. Two final questions. One, if you and I sitting down having this conversation in ten years time, what is your vision for the conversation that we're having? You know, what is happening in classrooms around the world, the way that the systems have perhaps supported more of the human elements of learning and all the humans doing the work.
00:36:23:04 - 00:36:50:05
Dylan Wiliam
My vision is that we can actually change the education system so that the people leaving school at 18, excuse me at 18 do so with the same passion and curiosity for learning that they had when they start at the age of 7. You know, we used to think we could teach kids everything they needed to know for the rest of their lives by the age of 18.
00:36:50:11 - 00:37:09:02
Dylan Wiliam
We now know that's impossible. They've got to continue to learn. And the one thing we can do that will really screw kids up is to extinguish that passion for learning that every human has. So for me, I think that, yes, we need high levels of achievement, we need high levels of skill in our young people, but we need to keep a broad and balanced curriculum.
00:37:09:04 - 00:37:38:05
Dylan Wiliam
We can't afford for demand for maths and science and English or mother tongue to squeeze out art and music and dance drama abroad and balance curricula. We have no idea. I mean, my favorite example of this right now is the top job in New York at the moment in terms of recruitment is a prompt engineer. Now these people who know how to write prompts to get the best out of Chat GPT, they're walking into school, they're working two jobs, paying $335,000 a year.
00:37:38:07 - 00:37:39:20
Louka Parry
Oh, my goodness.
00:37:39:22 - 00:38:07:19
Dylan Wiliam
Who's getting these jobs? No programmers. No programmers. English majors. English majors are walking into new jobs because they know how to actually use the English language to get the best out of Chat GPT. We have no idea what's coming. The the thing we can really do is: A) screw things up by killing the passion for learning and B) screw things up by, focusing our children's education narrowly because we think we know better.
00:38:07:21 - 00:38:11:23
Dylan Wiliam
You know, five years ago politicians were saying we have to teach coding.
00:38:12:00 - 00:38:14:05
Louka Parry
Yes, well, remember that.
00:38:14:07 - 00:38:16:11
Dylan Wiliam
The new coding language is English.
00:38:16:11 - 00:38:46:17
Louka Parry
Yeah, that's brilliant. I've never heard it put that way for those. Absolutely true. Oh, my goodness. Dylan Wiliam has been absolutely wonderful to speak with you, my final question is, what is this incredibly full, abundant, resonant conversation? What's a take home message that you would like people to take with them in the work that they do across schools as parents and anyone that's contributing in this way from this conversation?
00:38:46:19 - 00:39:12:10
Dylan Wiliam
I just think it's just don't lose your enthusiasm. I don't know whether Winston Churchill ever said this, but he's attributed to saying success means going from failure to failure without loss of enthusiasm. The best teachers never give up. They fail because they want every child to succeed. And you never get to the point where every child gets 100% on every test or every assessment you set.
00:39:12:12 - 00:39:36:10
Dylan Wiliam
But as long as you keep that goal in mind, we never get discouraged. The teachers have a growth mindset as well as giving their students so that you see, how can I be a better teacher next month than I was this month? Just keeping on bouncing that just going forward and forward, just never giving up because you know that if you carry on pushing that you will make a real difference in the lives of the young people that you serve.
00:39:36:12 - 00:39:49:24
Louka Parry
Well, Dylan, thank you for this conversation. Thank you for the work you do as a researcher, advocate, communicator, thought leader, author. I mean, it's very inspiring just to hear you speak. It's been a real pleasure. Thank you.
00:39:50:01 - 00:39:51:08
Dylan Wiliam
You're very welcome.