The Australian Curriculum, Assessment and Reporting Authority (ACARA) has defended the online tests, saying the results from both data sets will be comparable, but they remain under fire.
Associate professor James Ladwig, of the University of Newcastle’s School of Education, has questioned the validity of the online test results.
“It is well known that student performances in online test situations are not equivalent to pen and paper tests,” he said.
“ACARA has not yet released any of the technical information needed to assess the performance of the online version. As a consequence it is impossible to know the degree to which the test is valid and equivalent to older versions.”
Dr Steven Lewis of Deakin University cautioned that a lack of comparability, whether perceived or actual, could jeopardise NAPLAN’s utility as a trusted means of comparison.
“Such a lack of comparability could mean that comparisons cannot be made between schools using different modes of testing in 2018, or between a single school’s year-to-year performance if the school has piloted the online delivery format,” he said.
“My research has shown the profound impact of NAPLAN data, and comparisons of these data, on how schooling is understood and practised by teachers, schools and systems. Unless there is transparency around the statistical procedures and experts used by ACARA to make the data ‘valid and comparable’, there cannot be a complete trust in the evidence and comparisons these data provide.
“To this end, I would call on ACARA to release this information to help forestall what is arguably a growing mistrust, amongst the public and education professionals alike, in relying on NAPLAN and standardised test data to inform teacher practice and student learning.”
Jihyun Lee is an associate professor at the UNSW School of Education, specialising in large-scale standardised testing, including NAPLAN.
“A concern has been raised about the comparability between online and pen-and-paper test results of NAPLAN,” she said.
“I was cautiously optimistic that the online testing would be successful given that the basic statistical methodology has been around since the 1980s. Large-scale online testing has proven to be valid in several overseas programs since 1990s.
“However, if ACARA were to release the NAPLAN results in two separate forms, this would suggest a failure to ensure comparability. An independent body of measurement experts should review the most recent NAPLAN data.”
This commentary comes following a flurry of criticism, culminating in Queensland Education Minister Grace Grace’s declaration today that a national NAPLAN review is the only way to maintain public confidence in the tests.