Towards NIR-VIS Masked Face Recognition

04/14/2021
by   Hang Du, et al.
0

Near-infrared to visible (NIR-VIS) face recognition is the most common case in heterogeneous face recognition, which aims to match a pair of face images captured from two different modalities. Existing deep learning based methods have made remarkable progress in NIR-VIS face recognition, while it encounters certain newly-emerged difficulties during the pandemic of COVID-19, since people are supposed to wear facial masks to cut off the spread of the virus. We define this task as NIR-VIS masked face recognition, and find it problematic with the masked face in the NIR probe image. First, the lack of masked face data is a challenging issue for the network training. Second, most of the facial parts (cheeks, mouth, nose etc.) are fully occluded by the mask, which leads to a large amount of loss of information. Third, the domain gap still exists in the remaining facial parts. In such scenario, the existing methods suffer from significant performance degradation caused by the above issues. In this paper, we aim to address the challenge of NIR-VIS masked face recognition from the perspectives of training data and training method. Specifically, we propose a novel heterogeneous training method to maximize the mutual information shared by the face representation of two domains with the help of semi-siamese networks. In addition, a 3D face reconstruction based approach is employed to synthesize masked face from the existing NIR image. Resorting to these practices, our solution provides the domain-invariant face representation which is also robust to the mask occlusion. Extensive experiments on three NIR-VIS face datasets demonstrate the effectiveness and cross-dataset-generalization capacity of our method.

READ FULL TEXT

page 1

page 3

03/04/2021

When Face Recognition Meets Occlusion: A New Benchmark

The existing face recognition datasets usually lack occlusion samples, w...
06/10/2021

Multi-Dataset Benchmarks for Masked Identification using Contrastive Representation Learning

The COVID-19 pandemic has drastically changed accepted norms globally. W...
12/08/2021

A study of Biases in Common Face Recognition Datasets

The performance of commonly used facial recognition datasets have been s...
07/16/2020

Semi-Siamese Training for Shallow Face Learning

Most existing public face datasets, such as MS-Celeb-1M and VGGFace2, pr...
05/10/2021

Multi-Agent Semi-Siamese Training for Long-tail and Shallow Face Learning

With the recent development of deep convolutional neural networks and la...
05/07/2021

Efficient Masked Face Recognition Method during the COVID-19 Pandemic

The coronavirus disease (COVID-19) is an unparalleled crisis leading to ...
07/01/2016

Sparse Graphical Representation based Discriminant Analysis for Heterogeneous Face Recognition

Face images captured in heterogeneous environments, e.g., sketches gener...