[Theme music starts]

ELIZABETH:

From Schwartz Media, I’m Elizabeth Kulas, this is 7am

An error at the Department of Human Services caused the original robo-debt algorithm to restart, issuing thousands of unchecked debt notices. Royce Kurmelovs on how the program operates and why the government persists with it - in spite of its flaws.

[Theme music ends]

Archival tape -- Unidentified woman #1:

“When I say robo-debt, what do you think?”

Archival tape -- Unidentified woman #2:

“It’s weird, I’m getting these, like, dystopian vibes in which our government is able to prey on the fear and vulnerability of individuals using a robot that lacks essential information.”

Archival tape -- Unidentified woman #1:

“Well, that’s completely wrong, because what we have is a government who’s preying upon the fears and vulnerabilities of the individual using an algorithm that lacks all the essential information.”

Archival tape -- Unidentified woman #2:

“Oh my God, how did I get that so wrong?”

Archival tape -- Unidentified Man #1:

“At the heart of the system is computer-driven analysis of tax office and centrelink records, leading to mass letter mail outs.”

Archival tape -- Unidentified Man #2:

“An automated system is taking out the human factor, just giving people a number without explaining how it’s been arrived at. It’s causing too much hardship and unfairness.”

ELIZABETH:

So, Royce, what's new in the robo-debt story?

ROYCE:

Yeah, so the headline, I guess, is that I was linked an email sent round the Department of Human Services compliance workforce. It was essentially talking about how, in late April this year, a batch of robo-debt notices that weren’t meant to be sent out hit the mail after someone made a mistake and took the program off pause.

ELIZABETH:

Royce Kurmelovs is a freelance journalist and author. He recently wrote about the robo-debt program for The Saturday Paper.

ROYCE:

So essentially, you've got a batch of debts released to the public with very little human oversight between April 29th and May 3rd this year. It’s my understanding that this batch of debts that were involved in this were never meant to be released in this way.

We don't have an exact number on how many debts were released in this five day period. But when the program was first introduced back in 2016, this program was chugging out about 20,000 debt notices a week. So given that this was left to run for about four or five days uninterrupted, we're talking potentially thousands of debt notices that were released.

ELIZABETH:

So basically, the Department of Human Services issued a series, potentially thousands, of additional debt notices under this system that they know is flawed, because an algorithm that was paused was accidentally turned back on?

ROYCE:

Yes. The actual algorithm was just designed to run on its own without any human input. It picked up a batch of debts that have stayed on file for, you know, years, basically. So those debts never went away. They've always been there, they remain there to this day, waiting for someone to come along and check the calculation, basically. Unpausing it, the process of debt collection just began.

What used to happen was, you had a compliance officer who was a veteran, someone who possibly had a law degree, who'd been around, who understood the system. They would work quietly behind the scenes to investigate the situation, because the algorithm would pick up a discrepancy, bring it up in the workflow software. This person would take the name and would then go away and contact employers, contact banks, and use the powers given to them under the Social Security Act to get the information.

This allowed them to check for any errors. And it allowed them to basically verify that this debt existed before anyone was even contacted. And this meant potentially thousands of people out there who would would never have to hear about this. And when they did contact you, you had a reasonably good sense that, yeah, okay, maybe something happened here.

ELIZABETH:

And then what happened?

ROYCE:

So you had the department, you know, 2015, get this bright idea to automate the recovery process, to institute this algorithm, to remove all human oversight and just allow this thing to churn through the debt notices and make you personally responsible for going away and getting the information you need to disprove this debt. Basically, it reverses the onus of proof. In doing so, they went from churning out twenty thousand debt notices a year to churning out twenty thousand a week.

So in 2016, you had this, you know, a seven fold increase in the number of debt notices that were issued under this program and it sparked this massive public outcry. You had Not My Debt movement form. You had people in the media talking about this program. You had people having debt collectors sent after them. Around Christmas 2016, when the entire core of the public service in Canberra were on leave and there was no one to appeal to.

Back in 2017, there was a Senate inquiry that investigated the robo-debt scheme and basically delivered this report that where its first recommendation was, the whole thing should be halted, human oversight should be reinstated, all the debts that had so far been issued under should be reviewed by someone. Now, most of that hasn't happened. As far as the department is concerned, they seem to have quietly paused this program and reinstated some checks and balances and human oversight in the process of identifying and raising these debts.

Archival tape -- Unidentified Man #1:

“Is this a case of where the state may have allegedly used newfangled technology to try to streamline cases like this and it went terribly wrong?”

Archival tape -- Unidentified Woman #1:

“That’s exactly what happened.”

ROYCE:

By way of background, I mean, like, this program was trialled before elsewhere with very similar results. So when the U.S. state of Michigan kind of experimented with the process of using an automated algorithm to identify welfare fraud, it flooded the state's courts with something like 30,000 separate appeals of people saying, ‘hey, I don't actually owe a debt, this is ridiculous’.

Archival tape -- Unidentified Man #1:

“The agency accused Michigan citizens of fraudulently seeking unemployment payments. But the review found the computer wrongly accused claimants in 93% of cases between October of 2013 and August of 2015.”

ROYCE:

And it forced the program to basically be brought back under human oversight. Now, the same thing is happening, or has happened here in Australia in 2016 with that original automated recovery process, except that's been tried on a national scale and the government department haven't really quite backed down.

ELIZABETH:

So to the algorithm itself, how does it work?

ROYCE:

So what it does, it does what's called income matching. It takes the figure that a person has reported to Centrelink over the lifetime that they're on a Social Security payment and the corresponding figure for the same period that they reported to the ATO. Where there's a discrepancy, the algorithm basically looks through and goes, right, there's a discrepancy here, here, here and here. That must be a debt, therefore, pay up.

Just as a point of fact, this algorithm wasn't capable of taking into account things like casualised work. If you're working a few contract hours each week, sometimes more, sometimes less, it’s not cleanly distributed, it would frequently include multiple employer names. I think in one instance, the department had counted one employer 16 times. So, there are many different ways that this algorithm could, you know, encounter error. And when it did, it would issue essentially a phantom debt.

ELIZABETH:

And just to be clear, this is the same algorithm, which was accidentally switched back on for a period of five days this year?

ROYCE:

Yes, this was the same flawed algorithm that was, you know, running in 2016 that was accidentally switched back on in April.

ELIZABETH:

We'll be right back.

[Advertisement]

ELIZABETH:

So Royce, you've discovered that potentially thousands of debt notices were issued after the government promised that it had shut off this early robo-debt algorithm, essentially because it was turned back on my mistake. What does the Department of Human Services say about this?

ROYCE:

When I contacted the department, they acknowledged the error happened and the issue was quickly identified by staff and debts were immediately repaused. Now, a spokesperson for the department said that arrangements were in place to prevent this from happening again and workaround measures were put in place. But it wasn't quite clear what the department has actually done to prevent a repeat, or what happened to the people who were given those debts.

Archival tape -- Unidentified Woman #1:

“I declare open this hearing of the Senate community affairs references committee into Centrelink’s compliance program.”

ELIZABETH:

This is all happening at the same time as a parliamentary inquiry is being held into this scheme. That can't be easy for the department.

ROYCE:

Yes, so I mean, you know, this area comes at a time when the department has been appearing before the Senate inquiry. Looking at how the department's compliance activities changed over time. From the department's perspective, the problems have always lay in the interaction between its staff and the average person.

ELIZABETH:

Hmm.

ROYCE:

And so you have, you know figures they mostly know from Services Australia now talking in front of the Senate committee about, well, here are the various stages of the online interface. We've made it much better. It's much nicer now. It's much more friendly.

Archival tape -- Unidentified Woman #1:

“There has been an improvement in the system since CUPI has become introduced. Obviously we can’t go back and change what happened at the beginning, and everyone can acknowledge that perhaps that wasn’t the best system, but there’s been improvements since CUPI.”

ROYCE:

And so you kind of see the culmination of that in the last two weeks. They're essentially engaging in an act of language policing. Liberal Party Senator Holly Hughes last week speaking to representers from the West Australian legal community...

Archival tape -- Unidentified Woman #1:

“...Referring to media reports about robo-debt, it’s not a term we would use…”

Archival tape -- Unidentified Woman #2:

“Well it’s been used in this room this morning.”

Archival tape -- Unidentified Woman #3:

“No no, can we not argue about the name?”

ROYCE:

Calling the phrase robo-debt a “media slogan”, and saying it causes anxiety and saying that we should move away from this into something else, into the kind of more, I guess, sterile bureaucratic jargon.

Archival tape -- Unidentified Woman #1:

“Moving on from that then, if we can fundamentally agree on what the process is…”

ELIZABETH:

… Rather than actually allowing people to say, the legal basis of this program is the thing that's actually at issue.

ROYCE:

Yeah. And so the problem with this is that, in focusing on the way the program is talked about, it doesn't actually address the underlying architecture by which the program operates. And that goes to the basic legal basis of requiring people who are told that they may or may not have a debt to go ahead and disprove that debt exists.

At the Senate hearing, it was pointed out that this goes against the best practices in debt collection within the private sector.

Archival tape -- Unidentified Woman #1:

“... Who actually focus on the person with the debt, try to discover the reason for the debt, you know, because poverty isn’t a crime and it shouldn’t be something you’re punished for.”

ELIZABETH:

So what about the people who work within the department, the people who are implementing it?

ROYCE:

A lot of the veteran compliance officers who worked on these teams have been moved on and put into other positions on the basis that this automated process would take over for them. When it didn't, then you had to introduce more oversight. So what you have are people being put into a situation where they're required to debts that they question the legal basis for, knowing that they can't give enough time to actually do the job properly and then having to deal with the fallout when someone calls in and says, ‘hey, what are you doing, this is terrible’.

ELIZABETH:

And is there data on what those within the DHS workforce think about the program?

ROYCE:

Yes, so the CPSU did a survey of their members who were working in compliance officers. And basically what they found is 78 percent thought the algorithm should be scrapped entirely and 95 percent wanted, you know, human oversight reinstated. And so coupled with this kind of real intense pressure to make these performance targets, what you have is a demoralized, casualised workforce who understands the problems of the system but aren't being listened to.

ELIZABETH:

Royce, I mean, you've got people who are involved in implementing the program making complaints and saying this isn't working. You've got public outcry. Why does the government persist with it?

ROYCE:

Well, it's kind of an open question why the government persists on this. And again, it depends whose perspective you're looking at this from. The coalition government basically has their eyes on the cash.

And when this was first proposed, it outlined to the government saying, hey, look, you know, we project there is over a billion dollars in, you know, Social Security overpayments that we think we can claw back through this automated measure. And they booked it before they even went through the process of collecting it. And because the coalition has been in this drive towards achieving budget surplus, they saw this as, you know, a really easy way to make income.

ELIZABETH:

And have they been right about that? When you actually look at the sums, is that correct?

ROYCE:

No. In terms of the administration of the program versus the money it brings in, the difference is about one hundred million dollars, at most. So, you know, you're paying a lot of money to get very little return for this and you're making a whole lot of people's lives worse as a result.

ELIZABETH:

And what’s likely to happen with the program next?

ROYCE:

So the program currently is subject to several legal challenges. There have been two by the Victorian Legal Aid. Meanwhile, there's also a class action being run by Gordon Lawyers, which is calling for people to sign on. Now, these are incredibly significant, because to date the department has been very reluctant to allow any kind of legal challenge to be appealed, for fear of opening up some kind of avenue to challenging the legal basis for this program.

If these succeed, what you're looking at is the undermining of the basis of this program that they've invested a lot of time and energy into producing and that they see as part of this broader organisational vision.

Archival tape -- Unidentified Woman #1:

“And it seems passing strange that such a massive scheme can be introduced, and now we’re trying to fix it as we go along. But we’ve got a massive backlog of people with problems that could have have been avoided.”

ELIZABETH:

Royce, thank you so much.

ROYCE:

Thank you for having me.

[Advertisement]

[Theme music starts]

ELIZABETH:

Elsewhere in the news, the front pages of newspapers across Australia were blacked out on Monday as part of “The Right to Know” campaign, which is calling on parliament to enshrine press freedom and whistleblower protections. Responding in Parliament, Scott Morrison said that while journalism is not a crime, quote “no-one in this country is above the law.”

And Australia’s new federal police commissioner, Reece Kershaw, has personally apologised to refugee footballer Hakeem al-Araibi for the months he spent behind bars in Thailand. Al-Araibi is a permanent resident of Australia, and was locked up in Bangkok over vandalism charges. In a Senate committee in Canberra on Monday, Kershaw acknowledged a trail of bureaucratic errors that led to Al-Arabi’s detention and said that the AFP is working closely with the department to, quote, “resolve the policy and legal questions arising from this matter.”

This is 7am, I’m Elizabeth Kulas. See you Wednesday.

[Theme music ends]