To save you the time of reading that post, the key point raised was:
John Hattie had admitted half the statistics in Visible Learning are wrong.
Within a couple of days I had a reply on the post from John Hattie himself suggesting that I was “Retweeting a lie.”
I decided that this might be an opportunity to directly engage with Hattie regarding the post and the issues raised. He agreed, and what followed was a great 60-minute chat about all things education.
But first to that blog post.
So did Hattie ever say half the statistics in Visible Learning are wrong? Well, not according to him.
“The message is that someone tweeted from a conference in London a few months ago, that apparently I said ‘half the statistics are wrong’ and I never said that,” Hattie says.
But that’s not to say there weren’t issues with the statistics. The main argument centres around something called the Common Language Effect Size, which allows you to compare effect sizes in simple terms. For example, Strategy A has a 90 per cent chance of working better than Strategy B.
And it is here where Hattie admits there were some errors, He says, “Unfortunately at the very last minute I put the wrong column up on the – as part of the confidence levels rather than the actual common language exercise.”
It wasn’t until about three years late when some students in Norway alerted Hattie to the errors that the correct data was added to reprints of Visible Learning.
But as and when the next edition is published Hattie says, “I’d take all that stuff out because it didn’t work [anyway].”
Hattie says that whilst “there are some minor errors” in Visible Learning – errors that he says he and others have picked up – these are corrected quickly. And though he is disappointed that such errors occur he is confident that it doesn’t change the overall message of the book, “Not one iota.”
No doubt debate will continue in educational circles and the blogosphere, but Hattie says, “The nature of academia is that you live off critique and so I thrive off that and so that’s why I’m happy to talk to you and anyone [but] I’m not going to tell you right now everything’s perfect.”
Hattie says he’s been doing this work and publishing Visible Learning work since about 1989 and what fascinates him is, that in the 20 or 30 years since he started working on it, “No one – not a single person – has critiqued the idea and come up with an alternative explanation for the data.”
Given it’s not every day you have the chance to sit down with one of the foremost educational researchers in the world, I took the opportunity to discuss some other issues in and around education.
The first issue I wanted to tackle was the Traditional versus Progressive teaching debate. Some read this as the Didactic versus Student Centred or sometimes the Knowledge versus Skills argument but either way, lines get drawn in the sand and teachers choose their respective side.
Hattie was recently appointed the Chair of the Australian Institute for Teaching and School Leadership (AITSL), an organistaion who have devised a set of National Teacher Standards, are piloting an innovative approach to teaching called Learning Frontiers and who on their website, showcase “best practice” by way of vignettes and short interviews with teachers.
This is interesting because Learning Frontiers, many of the standards, vignettes and interviews extol the virtues of inquiry or discovery based learning and even Learning Styles; strategies about which Hattie is on record as saying, “We have a whole rhetoric about discovery learning, constructivism, about learning styles that has got zero evidence for them anywhere.”
I wanted to know if there was a disconnect between what he, as the chair of AITSL believed to be true, and what AITSL were promoting as ‘best practice.’
The first thing Hattie wanted to do was clear up his position around constructivism. Whilst he agrees there is – of course – a constructivist theory of knowing, he says, “There is not a constructivist theory of teaching. And that’s a massive difference and every time I mention it I try and exploit that difference which no one picks up.”
He went on to elaborate that often teachers have to deliberately teach students in order for them to construct knowledge. “I have no difficulty with constructivist knowledge’s of learning, and knowing, but not teaching.”
To further press his point home, Hattie referred to the article Should There Be a Three-Strikes Rule Against Pure Discovery Learning? by Richard Mayer which described three major series of investigations of discovery learning and acquiring learning that showed despite it continues to failed it is still used.
Hattie says, “I think the problem is that, that people fail to make that distinction between the theories of knowledge such as Piaget and the teaching.”
So why are such approached included in AITSL’s website?
“I’m disappointed if there are things on the AITSL website that talk about learning styles,” he said, “They did a massive review in the last month or so going through and hopefully taking everything out that’s hinting [at these concepts]. We do know some of that stuff has a high likelihood of not working [and] they have taken that seriously and they have gone through and if anyone finds stuff that’s still left there we’d all be delighted to know.”
And the standards?
Hattie says, “I’m quite happy with the standards, I think they’re moving in the right direction and my whole line is that Visible Learning is about looking at the impact of your teaching not about the particular teaching methods.”
The standards have been devised to enhance teacher quality, the one factor that most people agree on is paramount to improving student outcomes. However, the definition of and how we should measure teacher quality is not so easy to agree on.
For example, I put to Hattie that by judging teachers by their student’s academic performance could mean that a “quality” teacher in one school might actually be a “poor” teacher in another.
To answer this, Hattie used the analogy of Novak Djokovic, he says, “If you take him from Wimbledon to Australia he may lose. But overall you’d say he’s a consistently high quality player who’s got a higher probability of winning the Australian open than I do.
But he doesn’t succeed everywhere, and so in the sense that you talk about high quality teaching and high quality teachers, the argument I’m supporting is that high quality teachers do tend to be systematic regardless of circumstance but of course we can have our bad days and we can have our good days.”
So what are some of the things that high quality teachers do? Well according to Hattie, I’m asking the wrong question. He says, “I want to move away from talking about what teachers do – and saying high quality teachers do things.
I think that’s making a terrible mistake because you’re confusing the correlates with the outcomes. Just because teachers who use say reciprocal teaching or direct instruction typically get higher effect doesn’t mean that everybody should use reciprocal teaching and direct instruction.”
Rather Hattie says we should define teacher quality by the impact on students, which he argues is not solely focused on academic outcomes. Impact includes issues such as wellbeing and retention rates. He believes that we should make schools more “inviting places” for kids to want to finish high school, and points to the fact that Australia’s retention rate to Year 12 is around 73 – 78 per cent and whilst he makes no apology for looking at achievement, he says, “That [retention rates] is a bigger problem than achievement effects.”
Yet despite Hattie being fully aware of the other factors at play when defining teacher quality, he is also acutely aware that there could be some school leaders who are not.
I asked him if he felt there was a danger that leaders, armed with his work, could narrow their focus to only consider academic outcomes. “Absolutely. Yes.” He replied. And what about schools that were overly simplistic in their use of his Visible Learning work?
“Yes, for example they’ve administered the pre-test when kids know nothing the get 0. They administer a test five weeks later with some subject matter in the vocabulary. They get high effect sizes. They use very narrow measurements and very poor measurements.”
If you’ve read my blog, you’d be aware that a particular hobby horse of mine is the “Class Size debate,” so I couldn’t let him go without tackling this one.
I put it to him that many of Visible Learning’s high effect size strategies - classroom discussion, better feedback, teacher – student relationships etc – are dependent on class size and as such a smaller class size should mean higher effect size.
But Hattie has long argued that even when teachers are presented with smaller classes, they do not change their methods. I suggested that this might be because there aren’t too many small classes out there, and as such teachers aren’t trained to work with small groups.
In an interview in 2011, Hattie remarked, “You know of course if we did things differently than class size would make a difference.”
So perhaps that’s the point. Perhaps we need to teach teachers how to work with smaller groups. But as Hattie says, “I’m with you, I’d love to see a series of studies sort of where teachers are taught, particularly teachers who are out there in schools – they go and get retrained on how to optimise the smaller classes. As far as I can find out, and I could be proved wrong – it’s never been done.”
To finish, as a new generation of teachers entered the class for the first time this year, I wanted to know what Hattie’s advice would be to them.
“Make sure in the first month every kid has a friend. It’s the best predictor of investment in learning that we know of. Schools can be incredibly lonely places for a lot of kids. And having a friend is really critical. If you want high achievement the first month is about friendship,” he said.
“The second thing I’d be saying is that constantly ask yourself every time you walk into the classroom those three questions; how do I know I’m going to have impact, what does impact mean today? How do I know the size of the impact? The answers for that doesn’t come from giving lots of tests – it’s being open minded to seeing your impact, listening to the students.”
Hattie finished by saying, “Those are the two things I think are the most powerful; those are the two things they often don’t get taught in teachers colleges. Those are the things when you look at great teachers – that’s what they worry about in their first month, in their first year.”
It was a wide-ranging interview, and you can read more on my blog at danhaesler.com/blog.