Saturday, 25 June, 2022
HomeWorldA secret algorithm is deciding who will die in America

A secret algorithm is deciding who will die in America

The US govt's emergency management agency FEMA has a new pandemic-prediction model that is yet to be released to the public.

Text Size:

I have written extensively on the dangers that arise when authorities rely on inscrutable algorithms to make important decisions, bypassing public conversations. All too often, their statistically careless models end up costing lives and livelihoods.
Sadly, this appears to be precisely what will happen with the Trump administration’s Covid-19 model.

Arizona Governor Doug Ducey announced plans to allow businesses to start reopening this week after consulting the Federal Emergency Management Agency’s new pandemic-prediction model, which has not been released to the public. The move came as a shock because a team of university experts — who had developed their own model on the state’s behalf — had advised waiting until the end of May, lest the hospital system be overwhelmed.

In my book, “Weapons of Math Destruction,” I identified three properties that make a predictive algorithm particularly dangerous: It must be important, secret and destructive. FEMA’s new model has them all. If it can persuade a governor to lift stay-at-home orders in the middle of a pandemic, it’s important. As of this writing, its details remain a secret. And while what will happen in Arizona remains to be seen, it has the potential to destroy many lives by justifying bad decisions.

Such models don’t arise accidentally. They pop up when people want to avoid a difficult conversation, especially one that involves historical struggle, money and complex decisions. Illustrative examples include a model that assessed public school teachers, and the “crime risk score” algorithms that decide, among other things, who gets jailed pre-trial. Both were deeply flawed statistical embarrassments intended to bypass tricky issues, such as what makes a good teacher and who deserves incarceration. Yet governments and companies keep using black-box models, because people — due to fear of math or lack of information –- so rarely challenge them.

If there was ever a subject that required a difficult public conversation, the response to Covid-19 is it. How many people are we willing to let die in order to keep businesses working? And which people will we let die? By creating a secret model to inform such decisions, the Trump administration taking these questions out of the public and scientific spheres, replacing data-driven ethical debate with a pseudo-mathematical political tool. This is bad news for science, and potentially terrible news for Arizona residents.

Also read: Trump is a machine-learning algorithm gone wrong


Subscribe to our channels on YouTube & Telegram

Why news media is in crisis & How you can fix it

India needs free, fair, non-hyphenated and questioning journalism even more as it faces multiple crises.

But the news media is in a crisis of its own. There have been brutal layoffs and pay-cuts. The best of journalism is shrinking, yielding to crude prime-time spectacle.

ThePrint has the finest young reporters, columnists and editors working for it. Sustaining journalism of this quality needs smart and thinking people like you to pay for it. Whether you live in India or overseas, you can do it here.

Support Our Journalism

Most Popular