A deep feature-level fusion model for masked face identity recommendation system.

Researchers

Journal

Modalities

Models

Abstract

The widespread occurrences of airborne outbreaks (e.g., COVID-19) and pollution (e.g., PM2.5) have urged people in the affected regions to protect themselves by wearing face masks. In certain areas, wearing masks amidst such health-endangering times is even enforced by law. While most people wear masks to guard themselves against airborne substances, some exploit such excuses and use face masks to conceal their identity for criminal purposes such as shoplifting, robbery, drug transport, and assault. While automatic face recognition models have been proposed, most of these models aim to identify clear, unobstructed faces for authentication purposes and cannot effectively handle cases where masks cover most facial areas. To mitigate such a problem, this paper proposes a deep-learning-based feature-fusion framework, FIREC, that combines additional demographic-estimated features such as age, gender, and race into the underlying facial representation to compensate for the information lost due to mask obstruction. Given an image of a masked face, our system recommends a ranked list of potential identities of the person behind the mask. Empirical results show that the best configuration of our proposed framework can recognize bare faces and masked faces with the accuracy of 99.34% and 97.65% in terms of Hit@10, respectively. The proposed framework could greatly benefit high-recall facial identity recognition applications such as identifying potential suspects from CCTV or passers-by’s cameras, especially during crisis times when people commonly cover their faces with protective masks.© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2022, Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.