The impact of Covid-19 on facial recognition

The health crisis is changing our habits, and AIs weren’t expecting it. Find out how facial recognition is adapting to the COVID crisis.

Wearing a mask has become a very common action in the wake of the coronavirus epidemic. Like humans, facial recognition algorithms are hampered in identifying people. As these technologies become increasingly popular, like the latest generation of smartphones, these difficulties can erode public trust.

Masks to thwart facial recognition

A recent study has shown that facial recognition algorithms perform much less well when faced with masked people. A NIST report published at the end of July revealed an error rate of between 5% and 55%¹. Whereas the usual performance of an algorithm is around 0.3%. This is a major problem, but not a new one. Hong Kong demonstrators were already using this technique to fool facial recognition cameras. In response, the authorities imposed an outright ban on any means of concealing one’s face and evading facial recognition².

Here, the response to a technological failure is legislative. But this problem is not limited to state facial recognition systems. For example, it has become impossible for anyone to unlock their smartphone using facial recognition if they are wearing a mask. Apple, aware of the problem and the inconvenience caused by Face ID’s difficulty in unlocking devices, is trying to improve the situation. The minimum solution is already to detect whether the user is wearing a mask, and offer to unlock the device directly by typing in a code, rather than failing to identify the user³. The response here is to abandon the use of recognition as soon as it doesn’t take place under the expected conditions. Since it’s relatively easy to distinguish a face with a mask from one without, the parry works quickly, even if it sounds a little like an admission of failure.

How does facial recognition work?

To better understand why masks often prevent face recognition, we need to look at how these systems work. In the majority of cases, to identify a person, a software program will place points on certain areas of the face and calculate distances between them, which are specific to each individual.
The problem is that the areas of the face where markers are most numerous are the eyes, nose and mouth. If a mask hides the nose and mouth, the algorithm won’t have enough information to recognize a person effectively. The study also showed that the more a mask hides the nose, the less effective the algorithm will be⁴.

What are the solutions for improving facial recognition despite masks?

How does facial recognition work?

Several solutions are currently being tested to solve the problem. We have already discussed them in our article “Deliberately and physically misleading facial recognition“. One of them is to print your face on your mask. However, this technique can only work with 2D analysis algorithms⁵. For software like FaceID, which uses 3D beam projection, this seems complicated. It would then be necessary to have a mask reproducing the relief of one’s face, which would nevertheless be possible⁶ but perhaps not very pleasant to wear.

Alternatively, other companies are looking to develop solutions that identify people using only the areas around their eyes. It is also conceivable that the algorithms could be trained on databases of masked faces, which would improve performance by adapting the position of the markers. In many cases, we can see that the workarounds envisaged for the problem are not really technical in nature. The underlying technology can be adapted to the case of wearing a mask. However, for industry and governments, the adaptation time is still too long. Fast, immediate workarounds are needed. It is to be hoped that, in the longer term, more technological solutions will replace these makeshift countermeasures.

Besides the algorithmic methods that exist to trick these algorithms or infect their training bases, there are also concrete, physical methods. Some of these methods come close to the principle of masks as an annoyance. This subject will be covered in a future article!