Generalized Bilinear Deep Convolutional Neural Networks for Multimodal Biometric Identification

07/03/2018
by   Sobhan Soleymani, et al.
0

In this paper, we propose to employ a bank of modality-dedicated Convolutional Neural Networks (CNNs), fuse, train, and optimize them together for person classification tasks. A modality-dedicated CNN is used for each modality to extract modality-specific features. We demonstrate that, rather than spatial fusion at the convolutional layers, the fusion can be performed on the outputs of the fully-connected layers of the modality-specific CNNs without any loss of performance and with significant reduction in the number of parameters. We show that, using multiple CNNs with multimodal fusion at the feature-level, we significantly outperform systems that use unimodal representation. We study weighted feature, bilinear, and compact bilinear feature-level fusion algorithms for multimodal biometric person identification. Finally, We propose generalized compact bilinear fusion algorithm to deploy both the weighted feature fusion and compact bilinear schemes. We provide the results for the proposed algorithms on three challenging databases: CMU Multi-PIE, BioCop, and BIOMDATA.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/03/2018

Multi-Level Feature Abstraction from Convolutional Neural Networks for Multimodal Biometric Identification

In this paper, we propose a deep multimodal fusion network to fuse multi...
research
03/22/2018

Weighted Bilinear Coding over Salient Body Parts for Person Re-identification

Deep convolutional neural networks (CNNs) have demonstrated dominant per...
research
09/06/2022

Finger Multimodal Feature Fusion and Recognition Based on Channel Spatial Attention

Due to the instability and limitations of unimodal biometric systems, mu...
research
11/17/2016

Factorized Bilinear Models for Image Recognition

Although Deep Convolutional Neural Networks (CNNs) have liberated their ...
research
08/22/2018

CentralNet: a Multilayer Approach for Multimodal Fusion

This paper proposes a novel multimodal fusion approach, aiming to produc...
research
02/04/2020

Classification of Hyperspectral and LiDAR Data Using Coupled CNNs

In this paper, we propose an efficient and effective framework to fuse h...
research
12/16/2015

Multiregion Bilinear Convolutional Neural Networks for Person Re-Identification

In this work we propose a new architecture for person re-identification....

Please sign up or login with your details

Forgot password? Click here to reset