The Ethics of AI Decision-Making in Criminal Justice

Examining the ethical dilemmas of using AI in criminal justice, from predictive policing to sentencing algorithms.

Most of what you hear about AI in criminal justice is wrong. Not a little off. Deeply, structurally misleading. People talk about it like it’s some distant sci-fi scenario, a policy debate for think tanks and futurists. But right now — today — algorithms are helping decide who sits in jail and who walks free. Judges in courtrooms across the United States, the United Kingdom, Canada, and at least a dozen other countries consult AI-generated risk scores when they’re setting bail, handing down sentences, and deciding parole. Police departments point officers toward specific neighborhoods based on what a machine says will happen there. And the people on the receiving end? They’re overwhelmingly Black and brown, overwhelmingly poor, and they usually don’t even know a computer played a role in the outcome that upended their life.

I’ve covered the overlap between technology and civil liberties for over a decade. And I’ll say it plainly: AI in criminal justice is probably the most consequential and least understood use of artificial intelligence right now. It doesn’t generate viral images or say unhinged things in chatbot conversations, so it doesn’t grab headlines. But it’s deciding human freedom. We’re drifting into a world where that feels routine, and that should scare you.

What’s Actually Out There Right Now

Most people have no idea how much of this stuff is already deployed. So let me walk through it.

COMPAS — built by Equivant, which used to go by Northpointe — is a risk assessment tool that shows up in sentencing and parole decisions across most U.S. states. It pulls in over 130 data points about a defendant. Criminal history, employment, education, family background, substance use — all of it gets crunched into a score that supposedly predicts how likely someone is to reoffend. Judges get this score with their other case materials. Can they ignore it? Technically, sure. Do they? Studies say almost never.

PredPol is another one. They rebranded to Geolitica after the original name became too controversial (funny how that works). It’s a predictive policing system that chews through historical crime data and uses machine learning to spit out maps — little 500-by-500-foot boxes where crime is “most likely” to happen in the next 12 hours. Officers get sent to those boxes. At its peak, around 60 U.S. police departments used it, including the LAPD. A bunch have dropped it since, but competitor products carry on the same approach.

Facial recognition might be the scariest piece. Clearview AI scraped over 30 billion photos off the internet and social media to build a face-matching database. More than 3,100 law enforcement agencies in the U.S. alone use it. The FBI runs its own system, Next Generation Identification, with over 640 million face images in it. Over in London, the Metropolitan Police has been testing live facial recognition cameras at public events since 2020 — and they keep doing it despite repeated legal challenges and accuracy rates that independent reviewers have called “alarmingly poor.”

And those are just the big names. I’ve spent months looking into lesser-known tools: gang databases fueled by social media scraping, voice recognition systems that monitor inmate phone calls in prisons, AI-powered interrogation assistants that claim to read micro-expressions. The whole ecosystem is enormous, and most of it runs with barely any oversight.

Old Bias, New Packaging

I keep coming back to one problem, and I don’t think there’s a neat technical solution for it. Every single one of these systems learns from historical data. Historical crime data reflects decades — generations, really — of racially biased policing. If police spent more time in Black neighborhoods (they did), arrested Black people at higher rates for offenses that went unpoliced in white neighborhoods (they did), and prosecutors charged Black defendants harder (they did), then the data will “show” that Black people commit more crime. But that data isn’t measuring crime. It’s measuring where enforcement happened. Train an AI on enforcement patterns and you get a system that reproduces those patterns with eerie, automated precision, wearing a lab coat of objectivity while it does it.

ProPublica’s landmark 2016 investigation of COMPAS uncovered that it was roughly twice as likely to wrongly flag Black defendants as high-risk compared to white defendants. Equivant pushed back on the methodology, and there’s been a decade of academic argument about how to properly measure fairness. But nobody — not Equivant, not the critics, nobody — disputes that the system treats Black and white defendants differently, and the direction of that difference consistently hurts Black defendants. Whether you label it “bias” or a “fairness tradeoff” comes down to which statistical definition of fairness you pick. And the fact that multiple valid definitions exist? That’s part of the problem right there.

I visited a public defender’s office in Milwaukee in early 2026. Talked to lawyers who wrestle with COMPAS scores every day. One attorney, Sarah Chen — she’s been doing criminal defense for 22 years — said something I haven’t been able to shake. “My clients don’t get to see the algorithm. They don’t get to challenge the algorithm. They don’t get to know what inputs drove their score. They just get told they’re high-risk, and the judge nods and sets a higher bail. That’s not justice. I don’t even know what to call it, but it’s not justice.”

She’s right. I think she’s right, anyway. And what makes it worse is that this problem isn’t fixable by just “getting better data.” The data is a record of a broken system. You can’t unbreak history by feeding it into a machine.

The Feedback Loop Nobody Talks About

Predictive policing has an even more insidious issue, and it’s something that probably doesn’t get enough attention. The AI flags an area. Cops flood that area. They find crime — because when you concentrate enforcement anywhere, you concentrate arrests. Those fresh arrests feed back into the model as new data, confirming the original prediction. So the AI sends even more police next shift. Meanwhile, crimes in areas the algorithm didn’t flag go undetected, unrecorded, invisible to the system. The data gap widens. The cycle tightens.

Researchers at the Human Rights Data Analysis Group demonstrated this loop with hard numbers in a widely cited 2016 paper. Follow-up studies have confirmed it again and again. In 2025, a team at NYU’s AI Now Institute published an analysis of PredPol’s deployment in Oakland, California, and what they found was damning: the system directed officers disproportionately toward neighborhoods with high Black and Latino populations, even after controlling for actual crime rates. The disparity wasn’t subtle, either. Some predominantly minority neighborhoods got three to four times the patrol intensity of white neighborhoods with comparable crime statistics.

LA dropped PredPol in 2020. Oakland followed in 2021. New Orleans ditched its predictive policing program in 2022. But the idea keeps showing up in new forms. ShotSpotter, the gunshot detection system used in over 150 cities, works on a similar principle — it concentrates police attention in specific areas. Chicago’s Strategic Subject List, which people called the “heat list,” used AI to identify individuals supposedly most likely to be involved in gun violence. An internal audit found that 56% of Black men aged 20 to 29 in certain neighborhoods were on it. Fifty-six percent. That’s not prediction. That’s profiling wearing a tech hoodie.

When the Algorithm Gets the Wrong Face

Robert Williams. Remember that name, if you remember nothing else from this piece.

In January 2020, Williams — a Black man living in a suburb of Detroit — was arrested at his home. His wife watched. His two young daughters watched. The accusation: stealing watches from a Shinola store. The evidence: a facial recognition match generated by the Michigan State Police system, which compared surveillance footage against a driver’s license database. Wrong guy. Completely wrong. Williams spent 30 hours in custody before anyone acknowledged the mistake.

He wasn’t the last, either. Nijeer Parks in New Jersey — wrongly identified, jailed for ten days. Porcha Woodruff in Detroit — arrested while eight months pregnant, based on a faulty AI match. Randal Reid in Georgia — pulled over and detained 600 miles from home because an algorithm matched his face to a shoplifting suspect in a city he’d never visited. Every known case of a wrongful arrest from facial recognition in the United States has involved a Black person. Let that sit for a second. Every. Single. One.

A 2019 study by the National Institute of Standards and Technology tested 189 facial recognition algorithms from 99 developers. What they found should’ve stopped deployment cold: the best algorithms had false positive rates 10 to 100 times higher for Black and Asian faces compared to white faces. Some were especially bad with Black women, producing false matches at rates that made them useless — genuinely useless — for law enforcement. And yet police departments keep using them. Some don’t know about the accuracy problems. Some, I suspect, don’t care.

You Can’t Fight What You Can’t See

Maybe the part of all this that bothers me most is the secrecy. And I realize that’s saying something, given everything I just laid out. But think about it for a minute.

Most AI systems used in criminal justice are proprietary. The companies that build them claim trade secret protection. That means a defendant — a human being whose freedom hangs in the balance — can’t examine the algorithm that influenced their case. In 2016, the Wisconsin Supreme Court ruled in State v. Loomis that using COMPAS at sentencing didn’t violate due process, as long as it wasn’t the sole basis for the sentence. Defendants still couldn’t inspect it. That ruling basically gave a green light to black-box AI in courtrooms across the country.

Take a step back and consider what that means in practice. The Sixth Amendment guarantees you the right to confront evidence used against you. But when that evidence comes from an algorithm you can’t see, trained on data you can’t inspect, built with methods the company won’t disclose — what are you actually confronting? A number printed on a page. Try challenging that in a meaningful way when you’re represented by an overworked public defender who doesn’t have a data scientist on staff. Good luck with that.

Some places are trying to push back. New York City passed Local Law 144 in 2021, requiring bias audits of automated employment decision tools — though it doesn’t cover criminal justice applications, which seems like a pretty glaring omission. Illinois’ biometric privacy law, BIPA, has been used to challenge certain facial recognition practices. The EU’s AI Act, which rolled out in stages through 2025, classifies AI in law enforcement as “high-risk” and requires transparency and testing. But enforcement has been sluggish, and the United States still has no federal equivalent. Not one. We’re the country that deploys more of these systems than anyone, and we’ve got less regulation than Europe. Make of that what you will.

What This Actually Does to People

It’s easy to get lost in policy details and statistics. I do it all the time. So let me get concrete about what these systems mean for actual human beings, because I think that matters more than any technical analysis.

When a risk assessment algorithm labels someone high-risk, that person might sit in jail for months waiting for trial because they can’t afford the higher bail. During those months, they might lose their job. Their apartment. Custody of their kids. Even if they’re eventually acquitted — fully cleared — the damage is already done. You can’t un-lose your housing. A 2024 study from the Brennan Center for Justice found that pretrial detention, even for just three days, bumps up the likelihood of a guilty plea by 25%, regardless of whether the person actually did anything wrong. People plead guilty to get out. A biased algorithm helped put them in.

When predictive policing sends officers to the same blocks over and over, those neighborhoods don’t just see more arrests. They experience more stops. More searches. More confrontations. More trauma, accumulated over years and passed down to kids who grow up seeing police as something closer to an occupying force than a source of help. Trust breaks down so badly that people won’t report actual crimes — they don’t believe the system exists to help them. And that breakdown has a body count, measured in unsolved murders, unreported assaults, and whole communities that feel written off by the institutions that should be serving them.

I sat in a community meeting in the Bronx in late 2025. Residents were asked about police use of AI. A woman in her sixties — lived in the neighborhood her whole life — stood up and said, “They used to profile us with their eyes. Now they profile us with computers. The result is the same.” The whole room applauded. She wasn’t wrong.

Reform Is Possible. Probably. If We’re Serious About It.

I’m not going to sit here and tell you we should ban AI from criminal justice entirely, though I understand why some advocates push for exactly that. I’ve gone back and forth on it, honestly. These tools could do real good if someone actually built them right. Properly designed risk assessments might reduce pretrial detention by identifying genuinely low-risk defendants who’d otherwise sit in jail because a judge was being cautious. AI analysis of body camera footage could flag excessive force incidents. Pattern detection could help surface wrongful convictions that would otherwise stay buried.

But “properly designed” — those two words are doing an enormous amount of work in that sentence. For any of this to be ethical, several things have to happen, and they have to happen together.

First: mandatory transparency. If an AI system influences a decision about someone’s freedom, that person and their lawyer need to be able to examine the algorithm, the training data, and the specific inputs that produced the output. Trade secrets don’t get to override constitutional rights. Period. Second: independent bias audits, both before deployment and at regular intervals afterward, run by researchers who have zero financial connection to the vendor. Third: hard bans on the most dangerous uses — specifically live facial recognition in public spaces, which several cities have already enacted. Fourth: human override rules with actual teeth. Not “a judge can choose to ignore the score” but “no AI output can increase the severity of a disposition without independent human justification, documented on the record.”

Legislation exists that would address some of this. The Algorithmic Accountability Act, introduced in Congress in 2022 and reintroduced in 2024, would require impact assessments for automated systems used in critical decisions, including criminal justice. Hasn’t passed. The Justice in Forensic Algorithms Act would give defendants the right to access and challenge source code used against them. Also hasn’t passed. Both have bipartisan support — in principle — but keep dying in committee while the technology spreads to more jurisdictions every year. That pattern tells you something about where the priorities actually are.

Some Companies Drew a Line

Not every tech company has been eager to sell surveillance tools to police, and I think that’s worth noting even though the story is complicated. In June 2020, IBM pulled out of the facial recognition business entirely. Amazon put a moratorium on police use of its Rekognition platform. Microsoft pledged not to sell facial recognition to law enforcement until there was a federal law governing it.

Those were significant moves. But smaller companies — Clearview AI, Corsight AI, Rank One Computing, and others — rushed in to fill the gap. They don’t face the same public scrutiny or shareholder pressure that pushed the big names to step back. Axon, which makes Tasers and body cameras for police, announced in 2022 that it wouldn’t integrate facial recognition into its cameras after its own ethics board unanimously recommended against it. That board included civil rights leaders, AI researchers, and former law enforcement officials. It was one of those rare cases where an ethics board actually changed a business decision instead of just providing cover. But Axon is one company. The industry as a whole hasn’t shown much interest in policing itself. (Pun not intended, but I’ll leave it.)

Where I Am Right Now — And I Might Change My Mind

I’ve been wrestling with this stuff for years, and I want to be honest: I don’t have it all figured out. I’m still working through what I actually believe is the right path forward.

What I can tell you is this. The technology itself isn’t good or evil — an algorithm is a tool, same as any other. But a tool dropped into a system that has been systematically unfair for centuries will amplify that unfairness unless someone takes extraordinary care to prevent it. And right now, that care isn’t being taken. Not even close. The gap between what’s being deployed and what’s being regulated is enormous, and it’s growing.

The thing that nags at me the most — and I mean truly keeps me up some nights — is the veneer of objectivity these systems carry. When a human judge makes a biased call, you can name it. You can appeal it. You can look at their record and demand accountability. When an algorithm produces a biased result, it shows up wrapped in the language of math and probability, and people treat it like it’s neutral because they don’t understand how it works. The bias doesn’t vanish. It just gets harder to spot and harder to fight.

So look — if someone tells you AI will make criminal justice fairer, ask them some questions before you nod along. Whose data trained the model? Who audited it for bias? Can defendants see the code? Can they challenge it? Is there a human being who has to justify every AI-influenced decision in writing? If the answer to any of those is no, or “we’re working on it,” or “that’s proprietary information,” then you’re not looking at a fairness tool. You’re looking at an automation of how things already work.

And how things already work wasn’t doing right by millions of people. Making it faster and giving it a clean interface doesn’t fix that. It might actually make it harder to fix — which is the part that scares me, if I’m being honest.

I could be wrong about some of this. I’m still reading, still talking to people on every side. Ask me again in six months and maybe I’ll have a cleaner answer. But right now? Right now I think we’re building something dangerous, and we’re not building the safeguards fast enough to match. That’s where I land today. Tomorrow might be different, and I think that’s okay. These questions are too big for anyone to have locked down completely.