FingerNet: Pushing The Limits of Fingerprint Recognition Using Convolutional Neural Network

07/28/2019
by   Shervin Minaee, et al.
0

Fingerprint recognition has been utilized for cellphone authentication, airport security and beyond. Many different features and algorithms have been proposed to improve fingerprint recognition. In this paper, we propose an end-to-end deep learning framework for fingerprint recognition using convolutional neural networks (CNNs) which can jointly learn the feature representation and perform recognition. We train our model on a large-scale fingerprint recognition dataset, and improve over previous approaches in terms of accuracy. Our proposed model is able to achieve a very high recognition accuracy on a well-known fingerprint dataset. We believe this framework can be widely used for biometrics recognition tasks, making more scalable and accurate systems possible. We have also used a visualization technique to highlight the important areas in an input fingerprint image, that mostly impact the recognition results.

READ FULL TEXT VIEW PDF

Authors

page 1

page 3

page 4

07/22/2019

DeepIris: Iris Recognition Using A Deep Learning Approach

Iris recognition has been an active research area during last few decade...
03/21/2017

On the use of convolutional neural networks for robust classification of multiple fingerprint captures

Fingerprint classification is one of the most common approaches to accel...
05/27/2019

End-to-End Pore Extraction and Matching in Latent Fingerprints: Going Beyond Minutiae

Latent fingerprint recognition is not a new topic but it has attracted a...
09/10/2015

Real-time Sign Language Fingerspelling Recognition using Convolutional Neural Networks from Depth map

Sign language recognition is important for natural and convenient commun...
01/03/2018

Fingerprint Distortion Rectification using Deep Convolutional Neural Networks

Elastic distortion of fingerprints has a negative effect on the performa...
03/15/2021

Automatically Lock Your Neural Networks When You're Away

The smartphone and laptop can be unlocked by face or fingerprint recogni...
04/07/2021

A Unified Model for Fingerprint Authentication and Presentation Attack Detection

Typical fingerprint recognition systems are comprised of a spoof detecti...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

To make an application more secure and less accessible to undesired people, we need to be able to distinguish a person from the others. There are various ways to identify a person, and biometrics have been one of the most secure options so far. They are virtually impossible to imitate by anyone other than the desired person. They can be divided into two categories: behavioral features, which are actions that a person can uniquely create or express, such as signature and walking rhythm; and physiological features, which are characteristics that a person possesses, such as fingerprint and iris pattern. Many works revolve around recognition and categorization of such data including, but not limited to, fingerprints, faces, palmprints and iris patterns [1, 2, 3, 4, 5].

Fingerprint has been used in various applications such as forensics, transaction authentication, cellphone unlocking, etc. Many algorithms proposed for fingerprint recognition are minutiae-based matching. The major minutiae features of fingerprint ridges are ridge ending, bifurcation, and short ridge. There have also been various works on fingerprint recognition using hand-crafted features followed by some classification in the past. In [6], Park proposed a fingerprint recognition system based on SIFT features. In [7], Cappelli proposed a new representation based on a 3D data structure built from minutiae distances and angles called Minutiae Cylinder-Code (MCC). More recently, Minaee et al [8] proposed a fingerprint recognition using multi-layer scattering convolutional networks, which decomposes fingerprint images using wavelets of different scales and orientations.

Fig. 1: The images from the first (top) and second layers (bottom) of scattering transform [8], each image capturing the wavelet energies along specific orientation and scale.

Although many of the previous works achieve highly accurate performances, they involve a lot of pre-processing and using several hand-crafted features, which may not be optimal for different fingerprint datasets (collected under different conditions). In recent years, there have been a lot of focus on developing models for jointly learning the features, while doing prediction. Along this direction, convolutional neural networks (CNNs) [9]

have been very successful in various computer vision and natural language processing (NLP) tasks in recent years

[10]. Their success is mainly due to three factors: the availability of large-scale manually labeled datasets; powerful processing tools (such as GPGPUs); and good regularization techniques (such as dropout, etc.) that can prevent the overfitting problem.

Deep learning have been used for various problems such as classification, segmentation, super-resolution, image captioning, emotion analysis, face recognition, and object detection, and significantly improved the performance over traditional approaches

[11, 12, 13, 14, 15, 16, 17, 18, 19]

. It has also been used heavily for various NLP tasks, such as sentiment analysis, machine translation, name-entity-recognition, and question answering

[20, 21, 22, 23]

. More interestingly, it is shown that the features learned from some of these deep architectures can be transferred to other tasks easily, i.e. one can get the features from a trained model for a specific task and use it for a different task, by training a classifier/predictor on top of it

[24].

However, having a large-scale dataset (with several examples for each class of label) is crucial for the success of most of the current deep learning-based models. For fingerprint recognition, there are several public datasets with a reasonable size, but most of them come with a limited number of images per class (usually less than 20 fingerprint images per person), which makes it more challenging to train a convolutional neural network from scratch on these datasets.

In this work, we propose a deep learning framework for fingerprint recognition for the case where only a few samples are available for each class (few shots learning). It can get the fingerprint images and perform recognition directly. Previously there have been works using the features extracted from a pre-trained convolutional network

[25], and used along with various classifiers (such as SVM) to perform biometrics recognition, but in this work, we train a model for fingerprint recognition directly.

We train a convolutional neural network on one of the popular fingerprint datasets. We initialize the model’s weights with the ones trained on ImageNet dataset, and fine-tune it on our dataset. We also employ data augmentation techniques (such as flipping, random cropping, adding small amount of distortions) to increase the number of samples for each class. By doing so, we are able to achieve a very high recognition accuracy rate, on the test samples of this dataset. We believe this framework can be widely used for any other biometric recognition task.

The structure of the rest of this paper is as follows. Section II provides the the description of the overall proposed framework. Section III provides the experimental studies and comparison with previous works. And finally the paper is concluded in Section IV.

Ii The Proposed Framework

In this work, we propose a transfer learning approach by fine-tuning a pre-trained convolutional neural network on a popular fingerprint recognition dataset. We will first provide a quick introduction of transfer learning, and then discuss the proposed framework.

Ii-a Transfer Learning Approach

In transfer learning, a model trained on one task is re-purposed on another related task, usually by some adaptation toward the new task. For example, one can imagine using an image classification model trained on ImageNet [imagenet] to perform texture classification. It would be plausible to use the representation learned by a model, trained for general-purpose classification, for a different image processing task. There have been many works based on pre-trained deep learning models to perform a different task in the past few years.

There are two main ways in which the pre-trained model is used for a different task. In one approach, the pre-trained model, e.g. a language model, is treated as a feature extractor, and a classifier is trained on top of it to perform classification, e.g. sentiment analysis. Here the internal weights of the pre-trained model are not adapted to the new task.

In the other approach, the whole network, or a subset thereof, is fine-tuned on the new task. Therefore the pre-trained model weights are treated as the initial values for the new task, and are updated during the training stage.

Ii-B Fingerprint Image Classification Using Residual ConvNet

In this work, we focused on fingerprint recognition task, and chose a dataset with a large number of subjects, but limited number of images per subject, and proposed a transfer learning approach to perform identity recognition using a deep residual convolutional network. We use a pre-trained ResNet50 [11] model trained on ImageNet dataset, and fine-tune it on our own training images. ResNet is popular CNN architecture which provides easier gradient flow for more efficient training, and was the winner of the 2015 ImageNet competition. The core idea of ResNet is introducing a so-called identity shortcut connection that skips one or more layers, as shown in Figure 2. This would help the network to provide a direct path to the very early layers in the network, making the gradient updates for those layers much easier.

Fig. 2: The residual block used in ResNet model

To perform recognition on our fingerprint dataset, we fine-tuned a ResNet model with 50 layers on the augmented training set. The overall block diagram of ResNet50 model, and how it is used for fingerprint recognition is illustrated in Figure 3.

Fig. 3: The architecture of ResNet50 neural network [11], and how it is transferred for fingerprint recognition. The last layer is changed to match the number of classes in our dataset.

We applied several data augmentation techniques to increase the number of training samples, including horizontal and vertical flip, random crops, and small distortions. By adding these augmentation, the number of training samples is increased by a factor of

3x. We then fine-tune this model for a fixed number of epochs, which is determined based on the performance on a validation set, and then evaluate on the test set. This model is then trained with a cross-entropy loss function. To reduce the chance of over-fitting the

norm can be added to the loss function, resulting in an overall loss function as:

(1)

where is the cross-entropy loss, and

denotes the Frobenius norm of the weight matrix in the last layer. We can then minimize this loss function using stochastic gradient descent kind of algorithms.

Iii Experimental Results

In this section we provide the experimental results for the proposed algorithm, and the comparison with the previous works on this dataset.

We train the proposed model for 100 epochs using a Nvidia Tesla GPU. The batch size is set to 24, and Adam optimizer is used to optimize the loss function, with a learning rate of 0.0001. All images are down-sampled to 224x224 before being fed to the neural network. All our implementations are done in PyTorch

[26]. We present the details of the dataset used for our work in the next section, followed by quantitative and visual experimental results.

Iii-a Dataset

We evaluated our work using the PolyU fingerprint database which is provided by Hong Kong Polytechnic University [27, 28]. It contains 1480 images of 148 fingers. Six sample images from this dataset are shown in Figure 4. It can be seen that the fingerprint images in this dataset have slightly different color distributions, as well as different sizes.

Fig. 4: Six sample fingerprint images from PolyU dataset.

For each person, 4 images are used as test samples randomly, and the rest are used for training and validation.

Iii-B Comparison with previous works

Table I provides a comparison of the performance of the proposed model and some of the previous works on this dataset. As we can see, the model outperforms previous works on this dataset. It is worth mentioning that the model in [8] also uses features derived from a convolutional neural network with pre-defined filters. The scattering convolutional network can be thought of as a feature extractor trying to capture higher-order statistics.

Method Accuracy Rate
Scattering network [8]       92%
Gabor-wavelet [30] (on a subset of 50 subjects)       95.5%
The proposed algorithm       95.7%
TABLE I: Comparison of performance of different algorithms on PolyU database

Iii-C Important Regions Visualization

Here we use a simple approach to detect the most important regions while performing fingerprint recognition using convolutional network, originally inspired by [31]. We start from the top-left corner of the image, and each time zero out a square region of size x inside the image, and make a prediction using the trained model on the occluded image. If occluding that region causes the model to mis-classify that fingerprint image, that area would be considered as an important region while doing fingerprint recognition. On the other hand, if removing that region does not impact the model’s prediction, we infer that region is not as important. Now if we repeat this procedure for different sliding windows of x

, each time shifting them with a stride of

, we can get a saliency map for the most important regions in recognizing fingerprints. The saliency maps for four example fingerprint images are shown in Figure 5.

Fig. 5: The saliency map of important regions for fingerprint recognition.

Iv Conclusion

In this work we propose a deep learning framework for fingerprint recognition, by fine-tuning a pre-trained convolutional model on ImageNet. This framework is applicable for other biometrics recognition problems, and is especially useful for the cases where there are only a few labeled images available for each class. We apply the proposed framework on a well-known fingerprint dataset, PolyU, and achieved promising results, which outperforms previous approaches on this dataset. We train these models with very few original images per class. We also utilize a visualization technique to detect and highlight the most important regions of a fingerprint image during fingerprint recognition.

Acknowledgment

The authors would like to thank the biometric research group at PolyU Hong Kong for providing the fingerprint dataset used in this work. We would also like to thank Facebook AI research for open sourcing the PyTorch package.

References

  • [1] Marasco, Emanuela, and Arun Ross. ”A survey on antispoofing schemes for fingerprint recognition systems.” ACM Computing Surveys (CSUR) 47.2 (2015): 28.
  • [2] Minaee, Shervin, and AmirAli Abdolrashidi. ”Highly accurate palmprint recognition using statistical and wavelet features.” Signal Processing and Signal Processing Education Workshop (SP/SPE), IEEE, 2015.
  • [3] Bowyer, Kevin W., and Mark J. Burge, eds. ”Handbook of iris recognition”. London, UK: Springer, 2016.
  • [4] Ding, Changxing, and Dacheng Tao. ”Robust face recognition via multimodal deep face representation.” IEEE Transactions on Multimedia 17.11 (2015): 2049-2058.
  • [5] S Minaee, A Abdolrashidi, and Y Wang. ”Face recognition using scattering convolutional network.” Signal Processing in Medicine and Biology Symposium (SPMB). IEEE, 2017.
  • [6] U. Park, S. Pankanti, AK. Jain, “Fingerprint verification using SIFT features,” SPIE Defense and Security Symposium, 2008.
  • [7] R. Cappelli, M. Ferrara, D. Maltoni, “Minutia cylinder-code: A new representation and matching technique for fingerprint recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010.
  • [8] S Minaee, A Abdolrashidi, and Y Wang. ”Iris recognition using scattering transform and textural features.” Signal Processing and Signal Processing Education Workshop, IEEE, 2015.
  • [9] LeCun, Yann, et al. ”Gradient-based learning applied to document recognition.” Proceedings of the IEEE 86.11: 2278-2324, 1998.
  • [10] A Krizhevsky, I Sutskever, GE Hinton, ”Imagenet classification with deep convolutional neural networks”, Advances in neural information processing systems, 2012.
  • [11]

    He, Kaiming, et al. ”Deep residual learning for image recognition.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

  • [12] Badrinarayanan, Vijay, Alex Kendall, and Roberto Cipolla. ”Segnet: A deep convolutional encoder-decoder architecture for image segmentation.” IEEE transactions on pattern analysis and machine intelligence 39.12: 2481-2495, 2017.
  • [13] Ren, S., He, K., Girshick, R., Sun, J. “Faster r-cnn: Towards real-time object detection with region proposal networks”, In Advances in neural information processing systems, 2015.
  • [14] Dong, Chao, et al. ”Learning a deep convolutional network for image super-resolution.” European conference on computer vision. Springer, Cham, 2014.
  • [15] Minaee, Shervin, and Amirali Abdolrashidi. ”Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network.” arXiv preprint arXiv:1902.01019, 2019.
  • [16] Sun, Yi, et al. ”Deep learning face representation by joint identification-verification.” Advances in neural information processing systems. 2014.
  • [17] Minaee, Shervin, et al. ”MTBI Identification From Diffusion MR Images Using Bag of Adversarial Visual Features.” IEEE transactions on medical imaging, 2019.
  • [18]

    Minaee, Shervin, et al. ”A deep unsupervised learning approach toward MTBI identification using diffusion MRI.” International Conference on Engineering in Medicine and Biology Society (EMBC), IEEE, 2018.

  • [19] Karimi-Bidhendi, Saeed, Faramarz Munshi, and Ashfaq Munshi. ”Scalable Classification of Univariate and Multivariate Time Series.” 2018 IEEE International Conference on Big Data (Big Data). IEEE, 2018.
  • [20] Kim, Yoon. ”Convolutional neural networks for sentence classification.”, Conference on Empirical Methods on Natural Language Processing, 2014.
  • [21] Severyn, Aliaksei, and Alessandro Moschitti. ”Learning to rank short text pairs with convolutional deep neural networks.” Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. ACM, 2015.
  • [22] S Minaee, Z Liu. ”Automatic question-answering using a deep similarity neural network.” Global Conference on Signal and Information Processing, IEEE, 2017.
  • [23] Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. ”Neural machine translation by jointly learning to align and translate.” arXiv preprint arXiv:1409.0473 (2014).
  • [24] AS Razavian, H Azizpour, et al. ”CNN features off-the-shelf: an astounding baseline for recognition.” IEEE conference on computer vision and pattern recognition workshops, 2014.
  • [25] S Minaee, A Abdolrashidi, Y Wang. ”An experimental study of deep convolutional features for iris recognition.” signal processing in medicine and biology symposium (SPMB), IEEE, 2016.
  • [26] https://pytorch.org/
  • [27] http://www.comp.polyu.edu.hk/ biometrics/HRF/HRF.htm
  • [28] Q Zhao, D Zhang, L Zhang, and N Luo, ”Adaptive Fingerprint Pore Modeling and Extraction,” Pattern Recognition, 2010
  • [29] Q. Zhao, L. Zhang, D. Zhang, N. Luo, “Direct pore matching for fingerprint recognition,” In Advances in Biometrics, pp. 597- 606. Springer Berlin Heidelberg, 2009.
  • [30] HM Jirandeh, H Sadeghi, MSJ Rad, ”High-Resolution Automated Fingerprint Recognition System (AFRS) based on Gabor wavelet and SVM”, International Journal of Scientific and Engineering Research, Volume 5, 2014 .
  • [31] M Zeiler, R Fergus. ”Visualizing and understanding convolutional networks.” European conference on computer vision, springer, Cham, 2014.