From Government Executive:
... In Australia, the government set off on a radical plan to reduce overpayment of government benefits in 2016. ...
The Australian government had been manually searching for overpayments in programs for retirees, people with disabilities and students, among others. The 2016 program used algorithms to search out overpayments and send the bills. Christened “Robodebt,” the algorithm checked each individual’s payment against the average income of people in similar circumstances. If the algorithm determined that the person was likely overpaid by the government, it generated a bill.
Robodebt allowed the government to review 20,000 cases per week, instead of the 20,000 cases per year in the manual system it replaced. Government officials no longer had to contact employers to obtain data on employment history and payroll amounts, and the government no longer had to prove an individual had been overpaid. Instead, individuals had to prove that they had received the correct amount. If individuals didn’t pay quickly, debt collectors went to work.
The government launched Robodebt fast and claimed credit for catching recipients who had benefited from mistakes in the system. But many people receiving the notices were distraught. They often had to come up with big payments in just a few weeks. Some people had to sell their cars or take out loans, which was a huge burden on some of the country’s neediest residents. Others drained their meager savings. At least three people committed suicide, a Royal Commission found in a devastating 1,000-page report.
An investigation revealed that some repayment notices were incorrect. Some simply were false. Moreover, a 2019 court challenge found that Robodebt had violated important provisions of Australian law.
In July 2023, the Royal Commission pointed to “Robodebt’s unfairness, probable illegality, and cruelty.” When problems surfaced along the way, the commission concluded, “the path taken was to double down, to go on the attack in the media against those who complained and to maintain the falsehood that in fact the system had not changed at all” from the previous system. ...
Could this happen here? To a great extent, it already is happening. Republicans, abetted by Social Security's Office of Inspector General, have long implied that all overpayments are the result of fraud and that the agency must be merciless in collecting these debts. Recently they have been blaming the agency for not creating overpayments automatically based upon data from payroll companies. Already, there is no statute of limitations, those informed of alleged overpayments are given no information about how the alleged overpayments occurred, and all benefit payments are seized until the desperate claimant asks for a repayment schedule. It's harsh by design despite the fact that the agency often has no basis in fact for asserting an overpayment and many overpayments are due to mistakes made by Social Security. The current attitude is that if the computer says there's an overpayment, there must be an overpayment. It could all get worse with artificial intelligence.