Multi-Modal Biometric-based Human Identification Using Deep Convolutional Siamese Neural Network

Abstract

Biometric recognition is a critical task in security control systems. Although face biometric has long been granted the most accepted and practical biometric for human recognition, it can be easily stolen and imitated. It also has challenges getting reliable facial information from the low-resolution camera. In contrast, a gait physical biometric has been recently used for recognition. It can be more complicated to replicate and can also be taken from reliable information from the poor-quality camera. However, human body recognition has remained a problem since the lack of full-body detail within a short distance. Moreover, the unimodal biometric system still has constraints with the intrinsic factors of each trait. Recently, a deep Convolutional Neural Network (deepCNN) has been firmly applied to many fields in recognition research. Nevertheless, it needs a lot of labelled data for training. Biometrics data acquisition and labelling for creating large-scale datasets are still problematic. In this thesis, we propose a multimodal approach by combining two biometrics using a deep Convolutional Neural Network with a distance learning based Siamese Neural Network for human recognition. The proposed network model learns discriminative spatio-temporal features from gait and facial features. The extracted features from the two biometrics are fused into a common feature space at the feature level and sensor level methods for multimodal recognition. This study conducted experiments on the publicly available CASIA-B gait dataset, Yale-B faces dataset and a walking videos dataset of 25 users. The proposed model achieves a 97.3 % classification accuracy with an 0.97 F1 score and a 0.004 Equal Error Rate (EER). The proposed SNN model also achieves a 90.4% True Positive Rate (TPR) on gait and 89.7 % TPR on face modality, and 98.4% TPR on the multimodal system. The experimental results demonstrate that the system can classify people by learned features on Gait energy (GE) and Low-resolution (LR) face images directly. The proposed multimodal recognition performance evaluation is compatible in comparison to other multimodal recognition methods.