“Summer melt” sounds like some sort of delicious icy treat, but as Professor Tim Renick talks further, it quickly becomes clear that this is not a palatable concept.
Professor Renick is visiting New Zealand from Georgia State University (GSU) in the United States, where he has helped turn around graduation rates and eliminate achievement gaps based on ethnicity and income. He was the keynote speaker at the Tertiary Education Commission’s learner success conference, Ōritetanga Tertiary Success for Everyone, held this week in Auckland.
As it turns out, “summer melt” is what can happen between high school and tertiary education. It’s when the student has completed high school, been accepted to university, got the necessary grades – but something happens in between and they don’t show up. They don’t even make it to day one of their tertiary education. They effectively “melt” away.
So, Renick led an analysis of all the bureaucratic obstacles faced by students in the summer before they enrol. They found that filling in financial aid forms, sourcing immunisation records, completing class registration, and so on, were barriers for many students, particularly those who didn’t have the support of family members who had been to university before and understood the process.
To counter this, Renick and his team implemented a ChatBot that, based on artificial intelligence, could answer the students’ questions about the process. They expected to get around 5,000 questions. They received over 201,000. Students had all their questions – often lodged in the small hours of the morning – answered and enrolments started increasing.
This is just one example Renick gives as the sort of initiative they implemented at GSU to help iron out some of the inequities plaguing the institution.
Just fifteen years ago it was 40% more likely you would graduate from GSU if you were white than if you were African American. There were achievement gaps based on ethnicity and socioeconomic background. But thanks to a systems-approach, often based on data, Renick and his team have managed to turn things around.
“For each of the last four years, our African-American students, our low-income students, our first-in-family students, our Hispanic students – they’ve all graduated at the rate of the student body overall,” says Renick.
Another initiative was a data project aimed at trying to understand why students were dropping out before completion. They looked at students’ results and activity six months prior to dropping out in an effort to understand the early warning signs. A comprehensive list of risk factors emerged helping to inform a better system of early support for students at risk of walking away from their studies.
Fifteen years ago GSU had 30% completion rate. Now it sits at 80%.
“What GSU has done, when it finds something that works, it doesn’t target that on a subset of our student population. The change will touch every single student,” says Renick.
Renick is quick to point out that the solutions have not been at the expense of cultural responsiveness. But he also wants to make clear that it isn’t enough to attempt to address problems that relate to a group of students by only working with that group. A much bolder approach is required, a systems change that happens at the university level.
“We need to accept our own complicity in the fact that some students are doing less well than others. Some of the problems are systemic. Some of the problems are hard-wired into the practices we have in the tertiary sector. By taking on those practices, by putting a mirror to ourselves and saying ‘we’re part of the problem’ – that was the breakthrough,” says Renick.
“We had programmes that were dedicated to our African-American students and our low-income students and we still had low completion rates and significant equity gaps. But when we began adding to those programmes by getting our house in order at the university level, by recognising that our systems and our bureaucracy and its confusing nature is part of the problem – that’s when we began to succeed.”
Renick says the good news from GSU’s experience is that these problems can be overcome.
“It needn’t take huge amounts of resource and it needn’t take generations- that in a relatively short period of time we can imagine, design and implement some university systems that are much more equitable.”
It’s tempting to look for the copy and paste solution to replicate what Renick has done at GSU and apply it to our own systems in Aotearoa.
And to an extent that might be possible. They beauty of technological and data-driven solutions is that they can be used to find and address the need.
Paora Ammunson, Deputy Chief Executive Ōritetanga, Tertiary Education Commission agrees New Zealand is in a good position to adopt its own versions of the sorts of solutions that have worked for GSU. There are pockets of it happening already.
Ammunson points to Wintec as a good example. Similar to the way a business would analyse ‘customer journey experience’, Wintec has used a combination of data analysis and student interviews to get a sense of where the points of attrition are and what the students are feeling about it.
This has enabled them to identify various personas among their student body with common traits and track their activity from pre-enrolment to enrolment to orientation to all the way through their student journey. They can then map out empirically what percentage of students will drop out at each phase, and from there develop an intervention that is more likely to work with that type of person.
Take this persona for example: a 25-40 year old Māori male, who has dependents, who has probably had a somewhat awkward historical relationship with the social justice system, who has left high school with Level 2 NCEA – maybe in the middle of Level 3, is from a low income background and has transport issues.
Of the 14,000 students at Wintec, there are a 1000 students who fit this particular persona. For every 100 of these who express an interest in enrolling at Wintec, 30 of them do not enrol. In the first month, another 30 drop out. By the end of the first year, you have 29 left.
“When you see it in the data and then you start to put your head around it and you triangulate it with your own knowledge of New Zealand, it just turns a whole lot of lights on,” says Ammunson.
Ammunson says we tend to use ethnicity to frame our solutions. And indeed there are a large number of programmes aimed at Māori and Pasifika students that would support this. Ammunson says while these are well-intentioned, it’s not being Māori that indicates you’re going to need a lot of help. Rather it’s things like being first in your family to go to tertiary education, looking after children, living two hours away from the place of learning, having less than 13 externally assessed NCEA credits, and getting a C grade in your major area in the first year of study. All these are risk factors that can apply irrespective of ethnicity.
Ammunson gives EIT (Eastern Institute of Technology) as another good example of an institution that has used data effectively. EIT found that a disproportionate number of students were dropping out in the second week of study, so they implemented an intervention – a short assignment due in week 2 that would indicate who was going to need some extra support. This intervention was not race-tagged, yet Maori achievement improved dramatically as a result.
Some of the issues EIT and Wintec identified are very similar to those identified by GSU. The same sorts of pre-enrolment bureaucratic barriers that were tripping people up in Georgia are tripping people up here. And the indicators of dropping out early in the course are similar too.
“I think we’ve reached the time in New Zealand where we can say ‘In a systemic way, what can we do to lift all the boats?’,”says Ammunson.
“For some providers the first step might be: spend time analysing your data much more deeply – much deeper than just Māori, non-Māori, Pasifika, non-Pasifika. Get your hands on historical data and look for those patterns that Georgia State University identified so well. Stop and examine everything you do, and examine it with a laser-like precision.”
In many ways the timing is perfect in New Zealand to start expanding on some of the good things that are already happening in pockets and rolling them out across the system. The recent reform to the vocational education system will see the polytechnic sector in a much better position for collaborating on these sorts of measures.
“The ROVE decision enables us to take the best of this stuff and apply it across the 16 polytechnics which will be tied together under the new Institute of Skills and Technology,” says Ammunson.
Data and student experiences drawn from across the regional hubs could start to paint a comprehensive picture of where the points of attrition lie, not just regionally, but nationally too.
The Tertiary Education Commission’s learner success conference is a sign that the wheels are already in motion and fast gaining traction in this direction. Sessions ranged from overseas experience, to local stories of success, to effective data use, to the secondary-tertiary interface, to individual student experiences.
Ammunson is confident we can continue to improve learner success in New Zealand. It’s about taking a good hard look at our systems, and being bold enough to make changes where the evidence is telling us that change is needed.
He sums this all up beautifully with a quote from Dr Ranginui Walker, as heard quoted by AUT’s Walter Fraser:
“We spend so much time obsessing about the health of the budgie that we forget to evaluate the soundness of the mineshaft”.