The ban comes after software called DataWorks Plus and human error led to the wrongful arrest of a Black man in Detroit. Police arrested Robert Williams after the software incorrectly matched his driver’s license photo to security footage from the scene of a burglary. The incident is the first known instance of a wrongful arrest in the US based on an incorrect facial recognition match, according to the American Civil Liberties Union (ACLU). However, research suggests most facial recognition software isn’t great at correctly identifying people of color. A 2018 MIT study, for instance, found that three commercially available programs incorrectly identified dark-skinned women as much as 34.7 percent of the time. 

“This is a crucial victory for our privacy rights and for people like Robert Williams, who have been arrested for crimes they didn’t commit because of a technology law enforcement shouldn’t be using,” said ACLU of Massachusetts executive director Carol Rose. “Lawmakers nationwide should follow suit and immediately stop law enforcement use of this technology. This surveillance technology is dangerous when right, and dangerous when wrong.”

It’s worth pointing out that once Boston Mayor Marty Walsh signs the bill into law, it won’t prevent federal agencies like the FBI from using the tech while conducting investigations in the city. Still, with cities like San Francisco and Boston limiting the use of the technology, other jurisdictions may do the same.