Where test items are engaging and targeted to multiple abilities, and teachers eagerly await testing day and the results which will follow immediately after.
Could this become a reality sooner than we think, with the rollout of NAPLAN online?
Or is the digital divide in Australia far too great for us to be considering a move away from pen and paper assessment?
After an unsuccessful attempt at rolling out NAPLAN online this year, The Australian Curriculum, Assessment and Reporting Authority (ACARA) remain unperturbed, and are looking forward to progressing with the model in 2018.
Dr Stanley Rabinowitz, general manager of assessment and reporting at ACARA, says moving NAPLAN online will make for an improved, faster and more precise test.
“It’s a better test,” he says, “on a paper test it’s hard to really test knowledge and application”.
“It’s much more, ‘do you know things?’ not ‘can you really apply it?’”
“When you have the electronic version, you can do more simulations, you can move away from just multiple-choice questions, so it’s much more what we really want students to know and be able to demonstrate.”
Rabinowitz says ACARA studies also show online tests are typically more engaging for students.
A common complaint regarding the present NAPLAN testing arrangement, is the fact that it takes so long for teachers to receive results, making it difficult to employ any kind of remediation to address student weaknesses.
Rabinowitz acknowledges this concern, but says it will be old news when tests move online.
“Right now, the results of May’s test are just going back to schools.
“Under NAPLAN online we can get the results back not within three months, but within three weeks, and even shorter as our systems get better.”
He says this will have a huge impact within the average classroom.
“Overall, the quicker you get results back, the quicker teachers are either confirming what they know, or learning new things … they didn’t realise about students,” he says.
“Especially for a Year 3 student, in three months there’s a lot of normal development.
“By the time these results come back, they may be a little out of date.
“The student may have learned some [new] things, or developed differently.
“Getting the results back within three weeks, or even sooner, over time, means the teachers are getting a snapshot of the students right now, not wondering if anything has developed over the last three months.”
Rabinowitz says for students, the psychological impact of immediate feedback will also give them a greater sense of urgency and relevance when completing the test.
And finally, by moving the test online, we can deviate from a one-size-fits-all testing approach where all students answer the same questions. “Right now we only have an hour or so to test each domain,” Rabinowitz explains.
“So you have a limited number of items you can use to test.
“And because most students are from the middle of the [abilities] distribution, we tend to place a lot of items in the middle and fewer on the top and bottom.
“That creates the most reliable test for the largest number of students.”
With the online test however, Rabinowitz says ACARA is using a tailored test design which is an adaptive test.
“What that means is that, dependent on how you perform on the first set of items, you either get harder or easier items.”
“This way, pretty much all the items you get in the 45 or 50 items we have, they’re all geared to your achievement level.
“That makes the test much more reliable, much more precise. So the online test really is better, faster, more precise.”
While the online testing model seems to tick a lot of boxes, not all in the education sector are in favour of change.
The Australian Education Union (AEU) is strongly opposing the move towards NAPLAN online, with concerns over equity and school readiness.
“This only serves to reinforce inequality in our classrooms,” AEU’s Federal President Correna Haythorpe said in a statement.
“Students from low socio-economic backgrounds will be disadvantaged. These results will not show us learning outcomes, they will show us whether or not a child has had access to technology and how proficient they are at using that technology.”
Speaking to Australian Teacher Magazine, Haythorpe says the union has written to state and territory education ministers asking them to stop the rollout of NAPLAN online, and engage in urgent discussions about issues that have been raised by teachers and principals in schools.
“These issues range from schools’ capacity around IT, technical support and resources from education departments in terms of administering the test, but also we think that this test serves to reinforce the inequality in our classroom,” she says.
“Because what we know is that many of our schools in low socioeconomic areas, are significantly disadvantaged in their resource gaps and their capacity to have the ICT infrastructure in place for these tests.”
Rabinowitz says ACARA is aware that not every school is ready to go online next year, so there are plans in place for a three-year opt-in program.
“In its first year we expect somewhere between maybe 25 or 50 per cent of schools to be online,” he says.
“We’re going through readiness testing and those 25 or 50 per cent will be the ones definitely ready to go online this year.
“The lessons we learn from that experience will allow us to prepare the rest of the schools, and we’ll also have time to build up their infrastructure. So the three-year opt-in period is an important mitigation,” he says.
“There are a lot of people asking, and it’s an important question, ‘what about the remote schools, or very remote schools?’ They don’t even have the broadband now’.
“We are piloting to have a portable server to be sent up to the school, so they can just connect it and be able to move online.”
The AEU however, is not appeased by this prospect.
“I think the trial has clearly highlighted some of the issues, particularly when you look at schools in regional and rural areas.
“Not only do they not have the infrastructure in place, but many of them don’t have reliable internet connections as well,” Haythorpe says.
“If you’ve got a situation where schools have different levels of access to technology, then it reinforces, for us, the inequality that our students face.
“And when we’re hearing stories from schools that, in order to participate in this trial, they had to drive around the neighbourhood and gather computers from other schools, clearly there’s a problem.”
While infrastructure can be purchased, tech skills take longer to acquire, particularly if students don’t have access to devices at home.
“…Students from low socio-economic areas often don’t have the same level of access to technology as others,” Haythorpe says.
“Putting these students in a situation where they must sit the test online is unfair and will simply serve to measure the different levels of technology-proficiency.”
In light of concerns over a digital divide, Rabinowitz asserts, “this is not a coding test”.
“We’re not testing intimate technology skills, these are the basic skills, drag and drop, typing on a keyboard, just simple skills that if you do anything with a computer, you’ll have these skills,” he says.
“Plus we have practice tests, so students can get online [and] can practice their skills, with actual items that show the range of technology they’ll need for it.”
Regardless of access to technology, many primary school principals have expressed concerns over their Year 3 students’ abilities to touch-type, and whether this will prevent them from completing longer written responses in the allocated time frame.
“Certainly some schools said their kids did very little because their typing skills aren’t up to it, pecking on a keyboard with one finger,” South Australian Area Schools Leaders Association president Chris Roberts told The Advertiser newspaper.
On a national level, Dennis Yarrington, president of the Australian Primary Principals Association (APPA) says principals are worried about an over-emphasis on touch-typing.
“Online writing testing would likely lead to increased teaching of touch typing and keyboard skills in the early years, thereby taking time and focus away from essential teaching and learning,” an APPA position statement says.
“Testing student writing online, particularly in Year 3, will see teachers employ a different set of pedagogical and assessment skills to that currently employed, an emphasis upon keyboard and touch-typing skills over creative writing skills, and a reduction in opportunity for younger students to develop the essential fine motor skills required for writing.”
Another common grievance around the creative portion of the test, is the concept of online marking. Are computers really equipped to make judgments on creative writing pieces? According to ACARA, they are.
“Our evidence shows, and I can point to many, many studies … that for NAPLAN-type writing, the computer can score, as well as, if not better than the typical human marker,” Rabinowitz says.
“What about very unusual writing? Humour, or just very creative types of writing?
“Well, humans don’t always do very well with that either,” he continues.
Rabinowitz says the artificial intelligence programs used for marking these pieces, are trained with over 1000 scripts.
“So they’ve seen pretty much everything that kids are doing.”
“If they see something they don’t recognise, it actually flags it and says ‘I don’t know what this is,’ and we can send it to a human writer.”
Until teachers are convinced on this point, ACARA has proposed that in 2018, every creative writing piece will be double-marked, once by a computer and again by a human.
“If there’s a difference, we’ll send it to a second human marker to resolve the difference.
“So what we recognise, is we have to prove this new technology is up to the standard that teachers are already able to provide, and I see that as an important confidence-building exercise, that double-marking mitigation,” Rabinowitz says.
While there are many considerations in what is a big step from paper to web-based assessment, Rabinowitz is positive when he talks about the future which could be in store for online testing.
“I think there are three things we can hope to have and some of them are a little more realistic than others,” he says.
“The first one is, the results can come back immediately, you don’t have to wait three weeks. So you take the test, you get immediate feedback, I think that [would be] empowering for students and teachers.”
He also says that while a portion of students are still completing the test on paper, we’re not able to experience the full “bells and whistles” the technology has to offer.
“We don’t have video, for example, we don’t have some of the real production-type things, simulations, that you can do in a test.
“So, there’s that possibility as we move forward, to have even more production-type items.”
And thirdly, while there are no plans for this at the moment, Rabinowitz says down the track there might be an option for NAPLAN to become ‘on demand’ , in a sense.
“It’s possible down the line that NAPLAN can be less of an event, and more when the student and the teacher believe they’re ready to take the test.
“Or they bank, they take the part they’re comfortable with, bank those scores, and then take other strands of knowledge as they develop them.
“There’s any number of things, because the technology is there all of the time,” he says.
In the meantime, Rabinowitz says ACARA will continue to prepare the test for partial rollout in 2018.
“Sometime early next year before the May event, we’ll have a dress rehearsal. Students and schools about a month or two before they actually take the test, can make sure that they’ve got their processes in place, the infrastructure in place, and really understand how to do the test,” he says.
“…We believe this is an important step.
“We’re taking into account all the readiness issues, all the equity issues, the fairness issues and we believe this will be a better experience for students and the teachers will have more immediate and more actionable results. And that’s what this program is for.”
All the while, the AEU will continue their fight to stop the rollout. “We have determined to oppose the move towards NAPLAN going online,” Haythorpe says.
“We believe it is a fundamentally flawed process and must not be implemented.
“There are far too many considerations that have come up as part of this trial, so we actually want the state and territory ministers to stop the process, and to consult with the profession about these issues, and to come up with solutions to them.”
What will the result be for teachers and students come May next year? Only time will tell.