Facial Recognition Is Getting Better At Seeing Past Masks 2022


Key Takeaways

New facial-recognition algorithms are almost 100% accurate at identifying masked faces.This tech could be used to “unmask” protesters.Police already abuse face-recognition, using it for mass-surveillance. Unsplash / Atoms

It's possible that your phone's face-unlock might finally be able to work while you're wearing a mask—just in time for the end of the pandemic (and maybe not so good for protesters).

Researchers have found that face-recognition algorithms have gotten a lot better at working with just the top of the face, thanks to developers tweaking their algorithms. That's great news for phone users, but bad news for privacy, and even safety, in some parts of the world. 

“Face recognition data can be prone to error, which can implicate people for crimes they haven't committed,” writes the Electronic Frontier Foundation (EFF). “Facial recognition software is particularly bad at recognizing African Americans and other ethnic minorities, women, and young people, often misidentifying or failing to identify them, [and] disparately impacting certain groups.”

Better Recognition

A study from the National Institute of Standards and Technology (NIST) looked at 65 face recognition algorithms provided after mid-March 2020. It then compared their effectiveness by digitally adding masks to faces, and doing before/after tests. To run the tests, NIST used border crossing photographs and photos of applicants for immigration benefits.

The result? The algorithms are getting better. “While a few pre-pandemic algorithms still remain within the most accurate on masked photos, some developers have submitted algorithms after the pandemic showing significantly improved accuracy and are now among the most accurate in our test," the report states.

The best algorithms managed to correctly identify almost all people (a failure rate of just 0.3% for mask-wearers). With high-coverage masks, the failure rate rose to just 5%. Even better, these algorithms falsely accepted “no more than 1 in 100,000 impostors.”

Running facial recognition on a bunch of photos, even tricky, poorly captured border-crossing photos, is different from the 3D facial maps generated by phone face-unlock systems, but still. This is a big improvement over the previous test done by NIST.

"Some developers have submitted algorithms after the pandemic showing significantly improved accuracy."

Good News, Bad News

Clearly this is good news for phone users. Face ID on the iPhone is something of a liability in COVID times. If you want to use your iPhone for contactless payment via Apple Pay, you first have to unlock the iPhone (by entering your passcode), then activate Apple Pay, and then authenticate once again. With better accuracy comes easier access to your protected data.

But this improvement in recognizing masked faces also has its downsides. Protesters often wear masks now, in part because law enforcement takes video and photographs of protests and demonstrations and uses face recognition to identify participants (plus, well, masks prevent the spread of COVID). In the U.K., famous for its blanket CCTV surveillance, live facial-recognition cameras are being deployed by London's Metropolitan Police. 

Getty Images / Matthew Horwood

Demonstrations are a legitimate form of protest, and recognized as such in democratic countries. And yet police in Baltimore used a private face-recognition company to identify citizens with outstanding arrest warrants during protests several years ago. 

Even when face recognition is deployed in public under the guise of convenience, law enforcement can't help but sniff around. In 2017, a Californian golf tournament used cameras to scan attendees and screen VIPs for access to restricted areas. The cameras “eliminated long wait times by accurately identifying media members and tournament staff all while keeping a lookout for known persons of interest to law enforcement by searching against state/local and national law enforcement databases, keeping potential threats away by alerting the appropriate authorities,” writes Sport Techie's Diamond Leung [emphasis added].

Presently, China is using a facial recognition system from Chinese mobile phone company Huawei to track and spy on Uighur Muslims. This includes an “Uighur alert” feature that identifies people by ethnicity, and flags them to police. In the wake of Black Lives Matter protests, it's easy to imagine some U.S. police forces deploying such ethnically targeted tech.

You Can't Have it Both Ways

We're well aware of the old tradeoff between security and convenience. It's convenient to have no password, or to use the name of your dog. But it's more secure to use a unique, complex (and hard-to-remember) passphrase. 

Biometrics are already problematic for general ID. It's easy to get a new credit card number if yours is stolen, for instance. But if your fingerprints are compromised, you're screwed. And at least fingerprints are easy to control. You can wear gloves, or just not touch something. Your face is out there in public, recordable by anyone. And now, even wearing a mask won't help.

At least you don't have to pull out a credit card to pay for your groceries.