Skip to main content
Springer Nature Link
Log in

A feature-level fusion based improved multimodal biometric recognition system using ear and profile face

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Ear based identity recognition subject to uncontrolled conditions such as illumination changes, pose variation, low contrast, partial occlusion and noise, is an active research area in the field of biometrics. Meanwhile, multimodal biometrics is becoming increasingly popular and offers improved performance due to the use of multiple sources of information. In this paper, a multimodal biometrics system is proposed based on the ear and profile face that not only alleviates the short-comings of ear biometrics but also improves the overall recognition rate. The ear and profile face modalities are first represented individually using the combination of two efficient local feature descriptors namely, local phase quantization (LPQ) and local directional patterns (LDP). These histogram-based local descriptors are then combined into a high-dimensional feature vector that preserves complementary information in both frequency and spatial domains. The PCA along with the z-score normalization technique is independently applied on each feature vector and the resultant reduced feature vectors are combined at the feature level. The kernel discriminative common vector (KDCV) approach is finally exploited over the combined feature set to derive more discriminative and non-linear features for the identification of individuals using kNN classifier. The effectiveness of the proposed model has been verified with the deep features derived from three popular pre-trained CNN models such as AlexNet, VGG16 and GoogleNet. Experimental results on two benchmark databases clearly show that the proposed approach achieves better performance than individual modality and other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Abate AF, Nappi M, Riccio D, De Marsico M (2007) Face, ear and fingerprint: Designing multibiometric architectures. In: 14th International Conference on Image Analysis and Processing, IEEE, pp 437–442

  • Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 12:2037–2041

    MATH  Google Scholar 

  • Alshazly H, Linse C, Barth E, Martinetz T (2019) Ensembles of deep learning models and transfer learning for ear recognition. Sensors 19(19):4139

    Google Scholar 

  • Al-Waisy AS, Qahwaji R, Ipson S, Al-Fahdawi S, Nagem TA (2018) A multi-biometric iris recognition system based on a deep learning approach. Pattern Anal Appl 21(3):783–802

    MathSciNet  Google Scholar 

  • Annapurani K, Sadiq M, Malathy C (2015) Fusion of shape of the ear and tragus-a unique feature extraction method for ear authentication system. Expert Syst Appl 42(1):649–656

    Google Scholar 

  • Basit A, Shoaib M (2014) A human ear recognition method using nonlinear curvelet feature subspace. Int J Comput Math 91(3):616–624

    MATH  Google Scholar 

  • Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Google Scholar 

  • Benzaoui A, Hadid A, Boukrouche A (2014) Ear biometric recognition using local texture descriptors. J Electron Imaging 23(5):053008

    Google Scholar 

  • Benzaoui A, Boukrouche A (2017) Ear recognition using local color texture descriptors from one sample image per person. In: 4th International Conference on Control. Decision and Information Technologies (CoDIT), IEEE, pp 0827–0832

  • Benzaoui A, Kheider A, Boukrouche A (2015) Ear description and recognition using elbp and wavelets. In: 2015 International Conference on Applied Research in Computer Science And Engineering (Icar), IEEE, pp 1–6

  • Bertillon A (1890) La photographie judiciaire: avec un appendice sur la classiication et l’identiication anthropométriques. Gauthier-Villars, Paris

  • Bokade GU, Kanphade RD (2019) Secure multimodal biometric authentication using face, palmprint and ear: a feature level fusion approach. In: 2019 10th International Conference on Computing. Communication and Networking Technologies (ICCCNT), IEEE, pp 1–5

  • Burge M and Burger W (1998) Using ear biometrics for passive identification. In: 14th International information security conference, pp 139–148

  • Cevikalp H, Neamtu M, Wilkes M, Barkana A (2005) Discriminative common vectors for face recognition. IEEE Trans Pattern Anal Mach Intell 27(1):4–13

    Google Scholar 

  • Cevikalp H, Neamtu M, Wilkes M (2006) Discriminative common vector method with kernels. IEEE Trans Neural Netw 17(6):1550–1565

    Google Scholar 

  • Chan TS, Kumar A (2012) Reliable ear identification using 2-d quadrature filters. Pattern Recognit Lett 33(14):1870–1881

    Google Scholar 

  • Chang K, Bowyer KW, Sarkar S, Victor B (2003) Comparison and combination of ear and face images in appearance-based biometrics. IEEE Trans Pattern Anal Mach Intell 25(9):1160–1165

    Google Scholar 

  • Choraś M (2005) Ear biometrics based on geometrical feature extraction. Electron Lett Comput Visi Image Anal 5(3):84–95

    Google Scholar 

  • Chowdhury DP, Bakshi S, Guo G, Sa PK (2018) On applicability of tunable filter bank based feature for ear biometrics: a study from constrained to unconstrained. J Med Syst 42(1):11

    Google Scholar 

  • Dodge S, Mounsef J, Karam L (2018) Unconstrained ear recognition using deep neural networks. IET Biom 7(3):207–214

    Google Scholar 

  • Emeršič Ž, Štepec D, Štruc V, Peer P (2017a) Training convolutional neural networks with limited training data for ear recognition in the wild. In: 12th International conference IEEE automatic face & gesture recognition, pp 987–994

  • Emeršič Ž, Štepec D, Štruc V, Peer P, George A, Ahmad A, Omar E, Boult TE, Safdaii R, Zhou Y, et al. (2017b) The unconstrained ear recognition challenge. In: International Joint Conference on Biometrics (IJCB), IEEE, pp 715–724

  • Emeršič Ž, Štruc V, Peer P (2017c) Ear recognition: more than a survey. Neurocomputing 255:26–39

    Google Scholar 

  • Emeršič Ž, Meden B, Peer P, Štruc V (2018) Evaluation and analysis of ear recognition models: performance, complexity and resource requirements. Neural Comput Appl 32:15785–15800

    Google Scholar 

  • Eyiokur FI, Yaman D, Ekenel HK (2017) Domain adaptation for ear recognition using deep convolutional neural networks. iet Biometrics 7(3):199–206

    Google Scholar 

  • Galdámez PL, Raveane W, Arrieta AG (2017) A brief review of the ear recognition process using deep neural networks. J Appl Logic 24:62–70

    MathSciNet  Google Scholar 

  • Guo Y, Xu Z (2008) Ear recognition using a new local matching approach. In: 15th IEEE International Conference on Image Processing, IEEE, pp 289–292

  • Hanmandlu M et al (2013) Robust ear based authentication using local principal independent components. Expert Syst Appl 40(16):6478–6490

    Google Scholar 

  • Hansley EE, Segundo MP, Sarkar S (2018) Employing fusion of learned and handcrafted features for unconstrained ear recognition. IET Biom 7(3):215–223

    Google Scholar 

  • Hassaballah M, Alshazly HA, Ali AA (2019) Ear recognition using local binary patterns: a comparative experimental study. Expert Syst Appl 118:182–200

    Google Scholar 

  • Hezil N, Boukrouche A (2017) Multimodal biometric recognition using human ear and palmprint. IET Biom. https://doi.org/10.1049/iet-bmt.2016.0072

    Article  Google Scholar 

  • Huang Z, Liu Y, Li X, Li J (2015) An adaptive bimodal recognition framework using sparse coding for face and ear. Pattern Recognit Lett 53:69–76

    Google Scholar 

  • Hurley DJ, Nixon MS, Carter JN (2005) Force field feature extraction for ear biometrics. Comput Vis Image Underst 98(3):491–512

    Google Scholar 

  • Iannarelli A (1989) Ear identification. Paramont Publishing, London

    Google Scholar 

  • Ibrahim MI, Nixon MS, Mahmoodi S (2010) Shaped wavelets for curvilinear structures for ear biometrics. Advances in visual computing. Springer, Berlin, pp 499–508

    Google Scholar 

  • Jabid T, Kabir MH, Chae O (2010) Local directional pattern (ldp) for face recognition. In: Consumer Electronics (ICCE), 2010 Digest of Technical Papers International Conference on, IEEE, pp 329–330

  • Jain AK, Nandakumar K, Ross A (2016) 50 years of biometric research: accomplishments, challenges, and opportunities. Pattern Recognit Lett 79:80–105

    Google Scholar 

  • Kannala J, Rahtu E (2012) Bsif: Binarized statistical image features. In: 21st International Conference on Pattern Recognition (ICPR), IEEE, pp 1363–1366

  • Kim KI, Jung K, Kim HJ (2002) Face recognition using kernel principal component analysis. IEEE Signal Process Lett 9(2):40–42

    Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Google Scholar 

  • Kumar A, Wu C (2012) Automated human identification using ear imaging. Pattern Recognit 45(3):956–968

    Google Scholar 

  • Liu Q, Huang R, Lu H, Ma S (2002) Face recognition using kernel-based fisher discriminant analysis. In: Fifth IEEE International Conference on Automatic Face and Gesture Recognition, IEEE, pp 197–201

  • Meraoumia A, Chitroub S, Bouridane A (2015) An automated ear identification system using gabor filter responses. In: 2015 IEEE 13th International New Circuits and Systems Conference (NEWCAS), IEEE, pp 1–4

  • Minaee S, Abdolrashidi A, Su H, Bennamoun M, Zhang D (2019) Biometric recognition using deep learning: a survey. arXiv:1912.00271

  • Ojansivu V, Heikkilä J (2008) Blur insensitive texture classification using local phase quantization. International conference on image and signal processing. Springer, Berlin, pp 236–243

    Google Scholar 

  • Olanrewaju L, Oyebiyi O, Misra S, Maskeliunas R, Damasevicius R (2020) Secure ear biometrics using circular kernel principal component analysis, chebyshev transform hashing and bose-chaudhuri-hocquenghem error-correcting codes. Signal Image Video Process 14:847–855

    Google Scholar 

  • Omara I, Wu X, Zhang H, Du Y, Zuo W (2017a) Learning pairwise svm on deep features for ear recognition. In: 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), IEEE, pp 341–346

  • Omara I, Xiao G, Amrani M, Yan Z, Zuo W (2017b) Deep features for efficient multi-biometric recognition with face and ear images. In: Ninth International Conference on Digital Image Processing (ICDIP 2017), International Society for Optics and Photonics, vol 10420, p 104200D

  • Pflug A, Busch C (2012) Ear biometrics: a survey of detection, feature extraction and recognition methods. Biom IET 1(2):114–129

    Google Scholar 

  • Pflug A, Paul PN, Busch C (2014) A comparative study on texture and surface descriptors for ear biometrics. In: International Carnahan Conference on Security Technology (ICCST), IEEE, pp 1–6

  • Priyadharshini RA, Arivazhagan S, Arun M (2020) A deep learning approach for person identification using ear biometrics. Appl Intell. https://doi.org/10.1007/s10489-020-01995-8

    Article  Google Scholar 

  • Rahtu E, Heikkilä J, Ojansivu V, Ahonen T (2012) Local phase quantization for blur-insensitive image analysis. Image Vis Comput 30(8):501–512

    Google Scholar 

  • Rathore R, Prakash S, Gupta P (2013) Efficient human recognition system using ear and profile face. In: IEEE Sixth International Conference on Biometrics: Theory. Applications and Systems (BTAS), IEEE, pp 1–6

  • Ross A, Jain A (2003) Information fusion in biometrics. Pattern Recognit Lett 24(13):2115–2125

    Google Scholar 

  • Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    MathSciNet  Google Scholar 

  • Sarangi PP, Mishra BP, Dehuri S (2018a) Multimodal biometric recognition using human ear and profile face. In: 2018 4th International Conference on Recent Advances in Information Technology (RAIT), IEEE, pp 1–6

  • Sarangi PP, Mishra B, Dehuri S (2017a) Ear recognition using pyramid histogram of orientation gradients. In: 4th International Conference on Signal Processing and Integrated Networks (SPIN), IEEE, pp 590–595

  • Sarangi PP, Mishra B, Dehuri S (2017b) Pyramid histogram of oriented gradients based human ear identification. Int Jnl Control Theory Apps 10(15):125–133

    Google Scholar 

  • Sarangi PP, Panda M, Mishra BP, Dehuri S (2017c) An automated ear localization technique based on modified hausdorff distance. Proceedings of International Conference on Computer Vision and Image Processing. Springer, Berlin, pp 229–240

    Google Scholar 

  • Sarangi PP, Mishra BSP, Dehuri S (2018b) Fusion of phog and ldp local descriptors for kernel-based ear biometric recognition. Multimed Tools Appl 78:9595–9623

    Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition

  • Štepec D, Emeršič Ž, Peer P, Štruc V (2020) Constellation-based deep ear recognition. Deep biometrics. Springer, Berlin, pp 161–190

    Google Scholar 

  • Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9

  • Toygar Ö, Alqaralleh E, Afaneh A (2018) Symmetric ear and profile face fusion for identical twins and non-twins recognition. Signal Image Video Process. https://doi.org/10.1007/s11760-018-1263-3

    Article  Google Scholar 

  • Turk M, Pentland A (1991) Eigenfaces for recognition. J Cognit Neurosci 3(1):71–86

    Google Scholar 

  • Victor B, Bowyer K, Sarka S (2002) An evaluation of face and ear biometrics. In: 16th International Conference on Pattern Recognition, IEEE, vol 1, pp 429–432

  • Wang Z, Yang J, Zhu Y (2019) Review of ear biometrics. Arch Comput Methods Eng. https://doi.org/10.1007/s11831-019-09376-2

    Article  Google Scholar 

  • Xiaona X, Yue Z, Xiuqin P (2008) Multimodal recognition fusing ear and profile face based on kpca. In: 2nd International Symposium on Systems and Control in Aerospace and Astronautics, IEEE, pp 1–5

  • Xu XN, Mu ZC, Yuan L (2007) Feature-level fusion method based on kfda for multimodal recognition fusing ear and profile face. Int Conf Wavel Anal Pattern Recognit 3:1306–1310

    Google Scholar 

  • Xu X, Mu Z (2007) Feature fusion method based on kcca for ear and profile face based multimodal recognition. In: IEEE International Conference on Automation and Logistics, IEEE, pp 620–623

  • Yan P, Bowyer KW (2007) Biometric recognition using 3d ear shape. IEEE Trans Pattern Anal Machine Intell 29(8):1297–1308

    Google Scholar 

  • Yuan L, Mu ZC, Xu XN (2007) Multimodal recognition based on face and ear. Int Conf Wavel Anal Pattern Recognit 3:1203–1207

    Google Scholar 

  • Yuan L, Mu Z, Liu Y (2006) Multimodal recognition using face profile and ear. In: 1st International Symposium on Systems and Control in Aerospace and Astronautics, IEEE, pp 5–pp

  • Zhang DD (2000) Introduction to biometrics. Automated biometrics. Springer, Berlin, pp 1–21

    Google Scholar 

  • Zhang HJ, Mu ZC, Qu W, Liu LM, Zhang CY (2005) A novel approach for ear recognition based on ICA and RDF network. Int Conf Mach Learn Cybern 7:4511–4515

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, India

    Partha Pratim Sarangi

  2. Department of Computer Science & Engineering, Malaviya National Institute of Technology Jaipur, Jaipur, India

    Deepak Ranjan Nayak

  3. Department of Master of Computer Applications, Seemanta Engineering College, Mayurbhanj, India

    Madhumita Panda

  4. Department of Computer Science & Engineering, Indian Institute of Information Technology, Design and Manufacturing, Kancheepuram, India

    Banshidhar Majhi

Authors
  1. Partha Pratim Sarangi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Deepak Ranjan Nayak
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Madhumita Panda
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Banshidhar Majhi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Deepak Ranjan Nayak.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sarangi, P.P., Nayak, D.R., Panda, M. et al. A feature-level fusion based improved multimodal biometric recognition system using ear and profile face. J Ambient Intell Human Comput 13, 1867–1898 (2022). https://doi.org/10.1007/s12652-021-02952-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-02952-0

Keywords