With issues surrounding moderation, consistency and ranking continuing to plague NCEA and National Standards, JUDE BARBACK considers some ideas touted to bring more relevancy, meaning and fairness to our national models of assessment.
Both NCEA (National Certificate of Educational Achievement) and National Standards, like perhaps any models of school assessment in any country, have advocates and opponents.
National Standards, implemented in 2010, has failed to gain broad acceptance in the primary sector for a number of reasons, ranging from a lack of consistency to forcing schools into unhealthy competition.
Meanwhile most now accept that NCEA, despite a rocky start, is now a much sturdier vehicle for secondary assessment. However, claims surrounding the limitations and flaws with NCEA continue to emerge, and New Zealand’s declining performance in the international Programme for International Study Assessment (PISA) rankings is also cause for concern.
In light of such doubts about the current systems, and in light of the way education is evolving, perhaps it is time we looked at school assessment in completely new and different ways, or at least started thinking along those lines. But if we were to change, what would we change, how, and why?
Problematic areas of NCEA and National Standards
NCEA replaced the more traditional exam-based qualifications which yielded percentages and A, B, C and D grades, with a system that awards credits for Not Achieved, Achieved, Merit, or Excellence, through a mix of internally and externally assessed standards. The move to standards-based assessment – which essentially measures students’ learning in relation to benchmarks of expected level of performance – saw a departure from a system that hinged on scaling and norm-based assessment and consequently failed a large proportion of those assessed.
While the introduction of achievement standards was a vast improvement on the crude and unwieldy unit standards which preceded them, the breaking down of subjects into small components – standards – that then had to be assessed consistently across all schools, remained problematic for many. It introduced flexibility for learners, which was generally perceived as a good thing, but with this came problems with maintaining quality and consistency across all schools.
Similarly, National Standards takes a standards-based approach, assessing Year 0–8 students ‘at’, ‘above’, ‘below’ or ‘well below’ the prescribed standards for reading, writing and mathematics. Each standard pulls together an array of components and teachers make judgements about a student’s work as a whole, rather than a single snapshot assessment.
Many see the overall teacher judgment (OTJ) system as the strength of National Standards. In other countries the emphasis is on tests, whereas in New Zealand, many inputs form the assessment of a child’s learning. In making their judgements on students’ work, teachers use whichever tests they think are most appropriate and combine these results with everyday assessments and their own observations.
The difficulty is that by endeavouring to veer away from assessment that is focused on meeting specific criteria, and keep standards broad, they risk becoming vague, and subject to different teacher opinions.
One teacher’s opinion of what might be acceptable to meet the standard, might not be shared with another’s. Children who are deemed to be ‘at’ the standard at one school, may be rated ‘below’ or ‘above’ at another. As with NCEA, ensuring consistency with National Standards is problematic.
Criticism of inconsistency in NCEA is amplified in part by the emphasis on internal assessment. From the outset, there were fears that internal assessment would lead to “gaming” – schools using it as an opportunity to enhance their students’ results.
Certainly, the Weekend Herald’s analysis of five years of NCEA results included an interactive graphic that revealed big gaps between the achievement levels of internal and external assessment, particularly for lower decile schools. Calculus students in decile one schools, for example, achieved 83 per cent of internal assessments at Level 3 in 2012, but only 34 per cent achieved the external assessments.
However, the Ministry of Education says internal assessments are thought to yield better results as a wider sample of student evidence could be used in making the final judgement on student achievement.
Ensuring consistency through national moderation is something the Ministry has strived hard to achieve.
In NCEA’s infancy, there were many concerns raised about the quality of marking, particularly for internal assessments. The proportion of internally assessed work subject to checks increased from three per cent in 2006 to 10 per cent in 2008 after a team of full-time moderators was employed.
Education Minister Hekia Parata recently described the moderation process for NCEA, which is driven by the New Zealand Qualifications Authority (NZQA), as “thorough”. A number of independent reviews have supported the integrity of NCEA assessment and the Minister has taken action when schools were found not to be meeting NZQA’s moderation standards.
Moderation of National Standards remains a hazier concept. It is not compulsory, although over 2,000 teachers attended 125 workshops for improving moderation this year, suggesting that schools are striving for consistency.
Some have expressed fears that in an effort to firm up National Standards, the Ministry will resort to a more test-based programme of assessment. Among them is leading educationalist Professor John Hattie.
“If we do not get OTJs correct, then the march towards replacing teacher judgements with tests will be strong and hard to resist. Heaven help us if we have a NCLB1, SATs2, or NAPLAN3. They are the hardest to get rid of, and a fear should be the profession defaulting to these seemingly easy options,” Hattie told the 2011 Symposium on Assessment and Learner Outcomes in Wellington.
However, the Progress and Consistency Tool (PaCT) has been pegged as the solution to a lack of consistency in National Standards. Available to schools this year, it is essentially an online tool that captures a series of teachers’ judgements on aspects of mathematics, reading and writing, turns that into a PaCT score range, and recommends an overall judgement which a teacher can confirm or review.
However, even with better moderation, concerns remain over the effect of league table-type comparisons. League tables have the undesired effect of driving schools to focus heavily on the aspects upon which they are assessed.
Different approaches to NCEA naturally produce different results, which would be fine if schools weren’t ranked publicly, and consequently judged on the outcomes.
The same argument can be applied to National Standards. While no one disputes the importance of reading, writing and mathematics, the temptation for a fiercely competitive school is to focus on these aspects at the expense of other components of a student’s education outside the realms of National Standards, such as science, ICT, physical education and the arts.
The temptation to mark lightly, even within the parameters of tightened moderation, is also driven by a desire to perform well on the league tables.
Time for an overhaul?
There is no denying education is changing. Technology and all it brings – portable devices, educational games, social media – is having a dramatic effect on education at all levels.
Further, what students should learn is evolving. While few would dispute the importance of reading, writing and numeracy, there is a distinct push to extend education beyond traditional subjects.
Claire Amos, deputy principal of Hobsonville Point Secondary School is a strong advocate.
“Of course reading and writing are important and the learning areas measured at secondary level are also fine subjects to explore, however I’m not sure they are still as relevant as they once were.
“Our current models of assessment are based around a long-standing model of education that was born out of the Industrial Age and was based on meeting the needs of workers of that age. Now, particularly in New Zealand, we are facing a very different workplace, a knowledge-based landscape that requires some similar skills, but also many other skills not necessarily captured by the literacy and numeracy focus at primary and a set of siloed subjects at secondary, either.”
If education is changing – in terms of what is learned and how it is learned – then it stands to good reason that assessment must keep pace as well.
Amos believes we need to rethink the whole notion of ‘assessment’ for both secondary and primary students. In Education Review’s ‘Sector Voices’ supplement, she outlines how we should do away with exams and tests which measure little more than the ability to memorise and recall information under stress, and instead make better use of digital technologies to gather data over time and capture and analyse learner skills across a range of subjects.
“What if we were to forgo examinations and instead poured our resources into an expansive national team of moderators who could provide both professional learning around measuring progressions of competencies as well as check-marking educator judgements of progress made?
“In a digital age, the notion of national and local educators pair-marking and giving feedback synchronously on rich multimedia, multi-subject learning portfolios is completely viable. Imagine a national assessment framework that was not just the same old subjects ‘anytime, anywhere’ but rather key competencies demonstrated ‘anytime, anywhere, anyhow’.”
There is plenty of evidence to support that schools are increasingly using digital technologies to enhance the way assessment is differentiated according to students’ needs and interests. The Ministry’s TKI (Te Kete Ipurangi) site is rich with examples of both primary and secondary schools using applications such as My Portfolio, VoiceThread and KnowledgeNet to present learning for assessment purposes. Given these trends, it seems likely assessment is destined for a more digital presence that transcends subject divides.
Making better use of the curriculum
One of the major outcomes of the NCEA overhaul that followed the State Services Commission review in 2006 was to link the achievement standards more closely to the national curriculum. This, in addition to a number of other tweaks and changes, has helped to establish a qualification that is generally perceived as more reliable.
The National Standards are also linked to The New Zealand Curriculum, however experts have suggested there needs to be better alignment.
The 2011 OECD Reviews of Evaluation and Assessment in Education – New Zealand suggests that there needs to be better linkages between the National Standards, the curriculum and assessment.
John Hattie also notes that National Standards are based on “years”, presuming all students of the same age can move towards the same expectation, while the curriculum is not based on years, but on a deeper notion of development.
“While there is a ’year’ base in National Standards and a progression base in curriculum there is an absurdity that schools are asked to reconcile – and the best way is to ignore one of these mandates,” said Hattie to the Wellington Symposium.
Amos believes the curriculum, with its five key learning competencies (thinking; using language, symbols, and texts; managing self; relating to others; participating and contributing) already provides a strong platform for an overhaul of our assessment systems.
There has always been an emphasis on the key competencies in terms of assessment. Certainly, ever since the development of The New Zealand Curriculum there has been much discussion of how to assess the competencies. In her 2007 paper Assessing Key Competencies Rosemary Hipkins confronts this matter head-on:
“When thinking about whether to assess key competencies, we need to consider which aspects of existing practice remain appropriate and which need to be rethought, reshaped, and/or replaced. It’s also very important to consider what we might want to achieve by assessing key competencies. That question creates a useful ’frame’ for all the other considerations.”
Amos believes we could take this further still. “What I believe we need to change is the focus. Rather than the learning areas being the dominant foreground image and the key competencies a blur in the background, I believe we need to change the depth of field and bring the competencies into sharp focus so the ’subject’ softens to simply provide a context for learning.”
Collaboration not competitiveness
In addition to such suggestions about rethinking what we assess and how we assess it, there have also been many calls to cease making the results of NCEA and National Standards publicly available.
Bali Haque in his book Changing our Secondary Schools (see review page 27) points out that many high-performing education systems like those operating in Finland and many Asian countries do not compare their schools through league tables, and instead of naming and shaming, focus their efforts on strengthening the entire system.
Competitiveness among schools has become deeply entrenched since Tomorrow’s Schools reforms and in any event will be a difficult notion to shake off. Haque acknowledges the unlikelihood of scrapping league tables, and discusses some interesting alternatives for how we should present assessment results to more accurately reflect a school’s effectiveness.
One suggestion is to calculate and present a school’s NCEA pass rate based on both participation (i.e. the number of students for whom it was at least technically possible to gain an NCEA certificate) and the roll (i.e. the number of students on the school roll on 1 July of the year in question). Such an approach would give results more context and prompt questions about what learning and assessment is taking place.
Haque also suggests the development of ‘value added measures’ to determine what value a school adds to its students between their entry and their exit from school, as a more useful alternative to crude comparisons in the form of league tables.
Many schools already have some form of value-added measure. A good example of this can be seen at Otumoetai Intermediate in Tauranga, which scooped the supreme award at last year’s Prime Minister’s Excellence in Education Awards. The school measures the value it adds to students in the two years between entry and exit from the school across a range of areas.
Arguably more important than the ideas themselves, is the fact that such conversations and blue-sky thinking is happening. When the time comes to take a fresh look at NCEA and National Standards, or a major overhaul of assessment is on the agenda, the sector will be poised for constructive consultation to help guide a very important aspect of education into the future.