What if I told you that data scientists could come up with an algorithmic crime prevention program that could detect and stop crime - even before it happens? You might say, that sounds pretty cool. Now what if I told you that our bleeding-edge Minority-Report-o-matic was going to be developed and administered by a Florida Sheriff’s Department? Uh oh.
Unfortunately that is precisely what has happened in Pasco County, north of Tampa [The Tampa Bay Times]. The police department there has built what sounds like a ‘pre-crime’ unit, where they use data on arrests, financial transaction, social media, etc. to identify likely future offenders. So far, so Spielberg. And then, in a crucial departure from the plot of Minority Report, the Sheriff just sends deputies to harass the hell out of people “until they move or sue”, in the words of one deputy.
A Case Study in Abuse of Statistics
The new Pasco County Sheriff stepped in with this idea in 2011, as a way to reduce property crime:
The agency, which has 650 sworn law enforcement officers and covers a county of roughly 500,000 residents, would use data to predict where future crimes were likely to take place and who was likely to commit them, Nocco told reporters. Then deputies would find those people and “take them out” — thwarting criminal activity before it happened.
The system by which those likely offenders are identified sounds…well, it doesn’t seem like the most unreasonable thing in the world. These steps all make some sense:
Potential prolific offenders are first identified using an algorithm the department invented that gives people scores based on their criminal records. People get points each time they’re arrested, even when the charges are dropped. They get points for merely being a suspect.
The manual says people’s scores are “enhanced” — it does not say by how much — if they miss court dates, violate their probation or appear in five or more police reports, even if they were listed as a witness or the victim…the Sheriff’s Office told the Times that a computer generates the scores and creates an initial pool of offenders every three months.
I think this is actually probably a good way to generate a pool of people who might be expected to have a higher-than-average chance of engaging in criminal activity. The problem is what happens next:
[Surveillance] involved “directed harassment,” former STAR team Cpl. Royce Rodgers said…Rodgers and his team would show up at people’s homes just to make them uncomfortable, he said. They didn’t always log the contacts in the agency’s official records. He recalled parking five patrol cars outside one target’s home all night and visiting some as many as six times in a single day. They would do the same to targets’ friends, relatives and other “associates,” he said.
…
If the targets, their family members or associates wouldn’t speak to deputies or answer questions, STAR team deputies were told to look for code enforcement violations like faded mailbox numbers, a forgotten bag of trash or overgrown grass, Rodgers said.…Rodgers said people sometimes would fail to pay the fine, which would result in a warrant being issued for their arrest.
…
In interviews with the Times, 21 families targeted by the program described deputies pounding on their doors at all hours of the day and night. Nearly half said deputies sometimes surrounded their homes, lined their streets with patrol cars or shined flashlights into their windows.
And here’s the twist: this ‘pre-crime’ program is no more or less than an excuse to harass citizens and create bogus arrests. While I noted that the Pasco program is probably “better than random”, it has some serious problems:
[Experts] noted that Pasco’s scoring system awards points based on arrests, which can reflect racially biased policing practices and doesn’t take into account whether charges were dropped or the person was acquitted. Some experts were concerned that people can get points for having been suspected of a crime. There are no rules for what makes someone a suspect.
To editorialize - I think we all know there are rules for what can make someone a suspect. Being the wrong color, speaking the wrong language, living in the wrong part of town - those all seem to be pretty common ways to end up on the Sheriff’s bad side. But the ‘algorithm’ allows a sort of prejudice-laundering, with the real facts of the matter hiding behind a surface layer of ‘points’ and ‘scores’.
Watch the Watchers
“Algorithms” so often, as in this case, provide a thinly-veiled way to skirt the law and common decency. The Pasco Sheriff surely knew it would be unacceptable to start harassing arbitrary random young minorities, but created a system that had precisely that effect. In a very different field, many of the big “innovations” in financial technology are aimed at using alternative sources of data to evade laws like the Equal Credit Opportunity Act [Wikipedia].
The Pasco experience illustrates an all-too-common issue with machine-learning-based decision processes: they are implemented by a combination of the naive, the malevolent, and the incompetent. If the Pasco Sheriff had asked for the input of the best criminologists or data scientists they would have responded that building a ‘pre-crime’ system was both absurd and unethical - and he would have simply turned around and contracted out the job out to someone less thoughtful or less scrupulous.
There’s really no cure for this other than vigilance - within an organization, ensuring that these mechanisms once set up are subject to some sort of impartial review. And you just plain need to have a disinterested expert available to evaluate the results - you wouldn’t trust a kid to grade his own homework, and you wouldn’t trust a trained seal to pilot a 747. The Pasco County program combines all the worst aspects of both.
At least the trained seal would make for good meme fodder.