I watched a movie recently called Mercy, and it stuck with me in a way I wasn’t expecting.

At its core, it presents a system designed to predict crime before it happens. Not based on guesswork, but on data—everything from financial struggles, personal messages, behavioral patterns, and past actions. It doesn’t just watch people. It understands patterns. Or at least, it tries to.

And the more I sat with it, the more I realized the uncomfortable question it was really asking:

What happens when prediction starts to feel like truth?

At first, the idea of the Mercy system didn’t feel entirely wrong to me. In fact, part of me understood the appeal. If a system could gather information humans would never be able to fully process, if it could connect dots across lives, patterns, and behaviors that usually go unnoticed—then maybe it could prevent harm before it happens.

Used correctly, it could be powerful. Not as a judge. Not as an authority. But as a tool.

Because the truth is, humans miss things. We overlook patterns. We get emotional, biased, distracted. An AI system, at least in theory, doesn’t do that. It doesn’t feel fear or anger or loyalty. It just processes.

But that’s also where things start to get complicated.

Because “just processing data” sounds neutral… until you realize how much of that data is still human interpretation. Messages taken out of context. Journal entries written in frustration. Jokes between friends. Financial stress misread as intent. A pattern that looks dangerous when viewed from the outside, but might mean something completely different when lived from the inside.

And suddenly, prediction doesn’t feel like clarity anymore. It feels like judgment.

One of the things that really stood out to me in Mercy was how easily systems like this could be manipulated. Evidence can be hidden. Data can be shaped. And once a system is trusted enough, people don’t just use it honestly—they start using it strategically. To protect themselves. To shift outcomes. To control narratives.

And when that happens, the system stops being purely about truth.

It becomes about power.

That’s the part that unsettles me the most.

Because even if a system like Mercy is highly accurate, even if it works 99.9% of the time, that remaining fraction still matters. Not just statistically—but humanly. Because those are real people, real consequences, real lives shaped by a prediction that may or may not have fully understood them.

And then there’s the fear factor.

The idea that people might start changing not just their actions, but their thoughts. Their messages. Their private expressions. Not because they’ve done something wrong—but because they’re afraid of how it might be interpreted.

That’s where things quietly shift.

From prevention… to surveillance.
From protection… to control.

What I keep coming back to is this tension:

I can see why a system like Mercy would be useful. I can even see why, in certain situations, it could save lives. But I also can’t ignore what happens when people stop being seen as complex individuals and start being seen as probabilities.

The ending of the movie actually gave me something to sit with, though. Because instead of treating the system as an unquestionable judge, it leaned toward something more human—it began working with people. Helping them understand, investigate, and even prove innocence when possible.

And maybe that’s where the real answer sits.

Not in removing human judgment. Not in removing AI prediction. But in forcing the two to work together in a way where neither becomes absolute.

Because once something becomes absolute, it stops asking questions.

And when questions stop… that’s when mistakes become invisible.