Facial recognition in times of pandemic

In recent months, several clients have expressed their concern about the possible negative effect of the use of masks on facial recognition systems. In order to study the real extent of this new situation and determine whether they may indeed pose a security threat, the NIST (National Institute of Standards and Technology) has issued a report evaluating the pre COVID-19 face recognition algorithms using modified databases in which masks have been synthetically included. The main objective is to quantify the variation in the systems’ performance. To this end, a 1:1 facial verification test scenario has been proposed in which the facial image to be evaluated contains a mask, while the reference image is kept without it.

Discover what is face recognition with our interactive guide and learn about the origins of the technology, use cases, current techniques and how it will evolve in the future.

Masks have been added to the facial images artificially, first calculating a series of key points on the face and then tracing a kind of mask with different shapes, percentage of occlusion and colours. Although the study points out the limitations that can arise from the analysis of the results generated with synthetic information, it is a representative starting point for the possible effect of this type of accessory.

Examples of synthetic images generated for re-evaluation

After analysing almost 90 facial recognition algorithms sent for testing within the Face Recognition Vendor Test (FRVT) programme, the following conclusions were obtained:

• False rejection rates ( genuine users labelled as impostors) have increased in all tests, varying from 0.3% in the best algorithms with an average occlusion rate to 5% in those tests where the area covered by the mask reaches 70% of the face. Increases in false rejection rate of more than 50% have been obtained, usually due to a higher case of errors in facial detection.
• False acceptance rates (impostor users labelled as genuine) remain at the same level and even show a slight improvement. This result is key in determining that mask use does not pose a threat to the security of facial recognition systems from the point of view of impostor user acceptance.
• Not all types of masks show the same influence on the results: the highest rates of increase in false rejection are obtained with masks that cover the face broadly (such as surgical masks) while with rounded masks (such as K95) these increases are smaller.
• Increased rates of error are more pronounced with black masks than with light blue masks. It is not clear what the justification for this difference is, although the synthetic nature of the data (where plain colours are used without considering textures) means it would be hasty to draw conclusions and a more in-depth analysis with real photos should be awaited.

Impact on false rejection rates comparing surgical mask assessment (red line) versus base case (no mask, black line).

The results of this work are revealing of the generalising power of facial recognition systems based on deep learning, since they are capable of maintaining the capacity to extract characteristics with strong discriminatory power even in situations where a very significant part of the information disappears (remains hidden under the mask).

Although in any case it is still recommended that facial recognition processes are carried out, providing as much information as possible (that is, eliminating elements that may cause occlusion such as sunglasses, masks or others). The NIST study at least provides confidence in the robustness of the algorithms with respect to false acceptance, without doubt the factor of greatest concern when implementing a biometric identity verification system. However, the task of improving false rejection performance remains to avoid an impact on the user experience caused by repeated unsuccessful attempts at legitimate verification.