Artificial intelligence is already changing policing and justice. A Manitoba law professor says the bigger question is whether humans understand (and remain accountable for) the technology they increasingly rely on.
By James Ham
“People worry that computers will get too smart and take over the world,” says Dr. Katie Szilagyi.
“The real problem is that they’re too stupid – and they’ve already changed the world.”
It is a memorable way of framing a challenge that is becoming increasingly difficult for governments, courts and police organizations to ignore.
Artificial intelligence is no longer a distant technological question. Generative AI systems are already being used across workplaces, large technology companies are developing tools specifically for policing and public-sector use, and organizations are being forced to consider how much responsibility they are prepared to place in the hands of automated systems.
Szilagyi, an Associate Professor at the University of Manitoba’s Faculty of Law who specializes in the intersection of technology, law and artificial intelligence, explored those questions during a presentation to Manitoba police leaders.
Her message was neither anti-technology nor rooted in science-fiction fears.
Instead, she urged organizations to understand what today’s artificial intelligence systems can – and cannot – actually do before relying on them in environments where accuracy, accountability and public trust matter.
“We don’t want our understanding of AI to be mediated by science fiction narratives,” she said.
Modern large language models, Szilagyi explained, are essentially designed to statistically predict what word is most likely to come next in a sentence – a capability that can produce remarkably convincing results without genuine human understanding behind them.
“You are not actually having a conversation with ChatGPT,” she explained. “It’s not talking to you. It’s generating the statistically most likely response to your prompt.”
That distinction becomes increasingly important as AI tools move from novelty into professional environments.
One of the risks Szilagyi emphasized is “automation bias” – the tendency for people to gradually place greater trust in automated systems than in their own judgment.
She compared the phenomenon to familiar technologies such as backup cameras and lane-assist systems. As technology proves reliable over time, people naturally begin depending on it.
“The more we rely on automation, the more likely we are to trust the machine over our own judgment,” she said.
Within policing and the justice system, that tendency can carry much greater consequences.
Szilagyi pointed to real-world examples of AI-generated misinformation, including lawyers submitting fictitious court cases created by ChatGPT and an Air Canada chatbot that incorrectly advised a customer about bereavement travel policies.
“The problem with AI-generated information is that it can be very close to being correct,” she noted.
That “almost right” problem becomes especially significant when the information contributes to a police report, legal argument, government decision or other process where errors can have serious consequences.
AI-assisted police reporting is one example of where those questions are already emerging.
Szilagyi referenced “Draft One,” a report-writing tool developed by Axon that uses body-worn camera audio to generate draft police reports.
Supporters of such technology argue it could reduce administrative workload and allow officers to spend more time on operational duties.
But Szilagyi cautioned that efficiency is only one part of the equation.
Organizations must also consider transparency, accountability, bias, accuracy and the degree to which meaningful human oversight remains part of the process.
“We know from decades of human-computer interaction research that automation bias and automation complacency are not things that might happen,” she said. “They are things that will happen.”
There is also a larger governance question.
As private technology companies develop tools that increasingly become part of policing, courts and government decision-making, Szilagyi encouraged public-sector leaders to consider who designs those systems, how they operate and whose interests ultimately shape them.
“When outside vendors provide tools that become part of policing or justice systems, corporate interests are now entering public processes,” she said.
None of that means artificial intelligence should be rejected.
“There are absolutely useful applications for these technologies,” Szilagyi acknowledged. “But we need to understand what they are actually doing before we hand over responsibility to them.”
That may ultimately be the central issue.
The debate surrounding artificial intelligence often focuses on what the technology is capable of doing. For policing, government and the justice system, an equally important question may be what humans remain responsible for doing themselves.
Szilagyi illustrated that point with the familiar story of four people named Everybody, Somebody, Anybody and Nobody. It’s a story about what happens when responsibility is assumed to belong to someone else.
Artificial intelligence introduces a modern version of the same dilemma.
“Where is the author? Where is the information?” she asked.
“How do we ensure accuracy, completeness, and correctness?”
Those questions extend well beyond any particular piece of software.
They speak to the larger challenge facing institutions as artificial intelligence becomes increasingly embedded in everyday work: how to embrace useful technology without surrendering the human judgment, transparency and accountability upon which public trust depends.
The conversation about artificial intelligence in policing and justice is no longer about what may happen someday.
It is about decisions being made now.