Skip to main content
Log in

Artificial intelligence-Enabled deep learning model for multimodal biometric fusion

  • 1238: Recent Advances in Biometrics Based on Biomedical Information
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The goal of information security is to prevent unauthorized access to data. There are several conventional ways to confirm user identity, such as using a password, user name, and keys. These conventional methods are rather limited; they can be stolen, lost, copied, or cracked. Because multimodal biometric identification systems are more secure and have higher recognition efficiency than unimodal biometric systems, they get attention. Single-modal biometric recognition systems perform poorly in real-world public security operations because of poor biometric data quality. Some of the drawbacks of current multimodal fusion methods include low generalization and single-level fusion. This study presents a novel multimodal biometric fusion model that significantly enhances accuracy and generalization through the power of artificial intelligence. Various fusion methods, encompassing pixel-level, feature-level, and score-level fusion, are seamlessly integrated through deep neural networks. At the pixel level, we employ spatial, channel, and intensity fusion strategies to optimize the fusion process. On the feature level, modality-specific branches and jointly optimized representation layers establish robust dependencies between modalities through backpropagation. Finally, intelligent fusion techniques, such as Rank-1 and modality evaluation, are harnessed to blend matching scores on the score level. To validate the model's effectiveness, we construct a virtual homogeneous multimodal dataset using simulated operational data. Experimental results showcase significant improvements compared to single-modal algorithms, with a remarkable 2.2 percentage point increase in accuracy achieved through multimodal feature fusion. The score fusion method surpasses single-modal algorithms by 3.5 percentage points, reaching an impressive retrieval accuracy of 99.6%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Data shall be made available on request.

References   

  1. Ö. Toygar, F. O. Babalola and Y. Bi̇ti̇ri̇m (2020) "FYO: A Novel Multimodal Vein Database With Palmar, Dorsal and Wrist Biometrics," in IEEE Access 8: 82461–8247. https://doi.org/10.1109/ACCESS.2020.2991475.

  2. Kumar P, Mukherjee S, Saini R, Kaushik P, Roy PP, Dogra DP (2019) Multimodal Gait Recognition With Inertial Sensor Data and Video Using Evolutionary Algorithm. IEEE Trans Fuzzy Syst 27(5):956–965. https://doi.org/10.1109/TFUZZ.2018.2870590

    Article  Google Scholar 

  3. Atenco JC, Moreno JC, Ramírez JM (2023) Deep Learning Convolutional Network for Bimodal Biometric Recognition with Information Fusion at Feature Level. IEEE Lat Am Trans 21(5):652–661. https://doi.org/10.1109/TLA.2023.10130837

    Article  Google Scholar 

  4. Yuan C, Jiao S, Sun X, Wu QMJ (2022) MFFFLD: A Multimodal-Feature-Fusion-Based Fingerprint Liveness Detection. IEEE Transactions on Cognitive and Developmental Systems 14(2):648–661. https://doi.org/10.1109/TCDS.2021.3062624

    Article  Google Scholar 

  5. Huang Y, Ma H, Wang M (2023) Multimodal Finger Recognition Based on Asymmetric Networks With Fused Similarity. IEEE Access 11:17497–17509. https://doi.org/10.1109/ACCESS.2023.3242984

    Article  Google Scholar 

  6. Hammad M, Liu Y, Wang K (2019) Multimodal Biometric Authentication Systems Using Convolution Neural Network Based on Different Level Fusion of ECG and Fingerprint. IEEE Access 7:26527–26542. https://doi.org/10.1109/ACCESS.2018.2886573

    Article  Google Scholar 

  7. Kanhangad V, Kumar A, Zhang D (2008) "Comments on “An Adaptive Multimodal Biometric Management Algorithm,” in IEEE Transactions on Systems, Man, and Cybernetics. Part C (Applications and Reviews) 38(6):841–843. https://doi.org/10.1109/TSMCC.2008.2001570

    Article  Google Scholar 

  8. Poh N et al (2009) Benchmarking Quality-Dependent and Cost-Sensitive Score-Level Multimodal Biometric Fusion Algorithms. IEEE Trans Inf Forensics Secur 4(4):849–866. https://doi.org/10.1109/TIFS.2009.2034885

    Article  Google Scholar 

  9. Snelick R, Uludag U, Mink A, Indovina M, Jain A (2005) Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems. IEEE Trans Pattern Anal Mach Intell 27(3):450–455. https://doi.org/10.1109/TPAMI.2005.57

    Article  Google Scholar 

  10. Poh N, Kittler J, Bourlai T (2010) Quality-Based Score Normalization With Device Qualitative Information for Multimodal Biometric Fusion. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 40(3):539–554. https://doi.org/10.1109/TSMCA.2010.2041660

    Article  Google Scholar 

  11. Shekhar S, Patel VM, Nasrabadi NM, Chellappa R (2014) Joint Sparse Representation for Robust Multimodal Biometrics Recognition. IEEE Trans Pattern Anal Mach Intell 36(1):113–126. https://doi.org/10.1109/TPAMI.2013.109

    Article  Google Scholar 

  12. Walia GS, Jain G, Bansal N, Singh K (2020) Adaptive Weighted Graph Approach to Generate Multimodal Cancelable Biometric Templates. IEEE Trans Inf Forensics Secur 15:1945–1958. https://doi.org/10.1109/TIFS.2019.2954779

    Article  Google Scholar 

  13. Monwar MM, Gavrilova ML (2009) “Multimodal Biometric System Using Rank-Level Fusion Approach,” in IEEE Transactions on Systems, Man, and Cybernetics. Part B (Cybernetics) 39(4):867–878. https://doi.org/10.1109/TSMCB.2008.2009071

    Article  Google Scholar 

  14. Haghighat M, Abdel-Mottaleb M, Alhalabi W (2016) Discriminant Correlation Analysis: Real-Time Feature Level Fusion for Multimodal Biometric Recognition. IEEE Trans Inf Forensics Secur 11(9):1984–1996. https://doi.org/10.1109/TIFS.2016.2569061

    Article  Google Scholar 

  15. Nguyen K, Denman S, Sridharan S, Fookes C (2015) Score-Level Multibiometric Fusion Based on Dempster-Shafer Theory Incorporating Uncertainty Factors. IEEE Transactions on Human-Machine Systems 45(1):132–140. https://doi.org/10.1109/THMS.2014.2361437

    Article  Google Scholar 

  16. Conti V, Militello C, Sorbello F, Vitabile S (2010) “A Frequency-based Approach for Features Fusion in Fingerprint and Iris Multimodal Biometric Identification Systems,” in IEEE Transactions on Systems, Man, and Cybernetics. Part C (Applications and Reviews) 40(4):384–395. https://doi.org/10.1109/TSMCC.2010.2045374

    Article  Google Scholar 

  17. Poh N, Windridge D, Mottl V, Tatarchuk A, Eliseyev A (2010) Addressing Missing Values in Kernel-Based Multimodal Biometric Fusion Using Neutral Point Substitution. IEEE Trans Inf Forensics Secur 5(3):461–469. https://doi.org/10.1109/TIFS.2010.2053535

    Article  Google Scholar 

  18. Zhang X, Cheng D, Jia P, Dai Y, Xu X (2020) An Efficient Android-Based Multimodal Biometric Authentication System With Face and Voice. IEEE Access 8:102757–102772. https://doi.org/10.1109/ACCESS.2020.2999115

    Article  Google Scholar 

  19. Guo BH, Nixon MS, Carter JN (2019) Soft Biometric Fusion for Subject Recognition at a Distance. IEEE Trans Biomet Behav Identity Sci 1(4):292–301. https://doi.org/10.1109/TBIOM.2019.2943934

    Article  Google Scholar 

  20. Parashar A, Parashar A, Abate AF, Shekhawat RS, Rida I (2023) Real-time gait biometrics for surveillance applications: A review. In Image and Vision Computing 138:104784. https://doi.org/10.1016/j.imavis.2023.104784

    Article  Google Scholar 

  21. Zhang H, Li S, Shi Y, Yang J (2019) Graph Fusion for Finger Multimodal Biometrics. IEEE Access 7:28607–28615. https://doi.org/10.1109/ACCESS.2019.2902133

    Article  Google Scholar 

  22. Veeramachaneni K, Osadciw LA, Varshney PK (2005) “An adaptive multimodal biometric management algorithm,” in IEEE Transactions on Systems, Man, and Cybernetics. Part C (Applications and Reviews) 35(3):344–356. https://doi.org/10.1109/TSMCC.2005.848191

    Article  Google Scholar 

  23. Sultana M, Paul PP, Gavrilova ML (2018) Social Behavioral Information Fusion in Multimodal Biometrics. IEEE Trans Syst Man Cybernetics: Systems 48(12):2176–2187. https://doi.org/10.1109/TSMC.2017.2690321

    Article  Google Scholar 

  24. Rida, I., Al-Maadeed, N., Al-Maadeed, S., & Bakshi, S. (2018). A comprehensive overview of feature representation for biometric recognition. In Multimedia Tools and Applications 79 7–8: 4867–4890. Springer Science and Business Media LLC. https://doi.org/10.1007/s11042-018-6808-5

  25. Ghaderi A, Shahri AA, Larsson S (2022) A visualized hybrid intelligent model to delineate Swedish fine-grained soil layers using clay sensitivity. CATENA 214:106289

    Article  Google Scholar 

  26. Abbaszadeh Shahri A, Shan C, Larsson S (2022) A novel approach to uncertainty quantification in groundwater table modeling by automated predictive deep learning. Nat Resour Res 31(3):1351–1373

    Article  Google Scholar 

  27. Naik DL, Kiran R (2021) A novel sensitivity-based method for feature selection. Journal of Big Data 8:1–16

    Article  Google Scholar 

  28. Asheghi R, Hosseini SA, Saneie M, Shahri AA (2020) Updating the neural network sediment load models using different sensitivity analysis methods: a regional application. J Hydroinf 22(3):562–577

    Article  Google Scholar 

  29. Paul PP, Gavrilova ML, Alhajj R (2014) Decision Fusion for Multimodal Biometrics Using Social Network Analysis. IEEE Trans Syst Man Cybern: Syst 44(11):1522–1533. https://doi.org/10.1109/TSMC.2014.2331920

    Article  Google Scholar 

  30. Bahrampour S, Nasrabadi NM, Ray A, Jenkins WK (2016) Multimodal Task-Driven Dictionary Learning for Image Classification. IEEE Trans Image Process 25(1):24–38. https://doi.org/10.1109/TIP.2015.2496275

    Article  MathSciNet  Google Scholar 

  31. Iula A, Micucci M (2022) Multimodal Biometric Recognition Based on 3D Ultrasound Palmprint-Hand Geometry Fusion. IEEE Access 10:7914–7925. https://doi.org/10.1109/ACCESS.2022.3143433

    Article  Google Scholar 

  32. Jiang RM, Sadka AH, Crookes D (2010) “Multimodal Biometric Human Recognition for Perceptual Human-Computer Interaction,” in IEEE Transactions on Systems, Man, and Cybernetics. Part C (Applications and Reviews) 40(6):676–681. https://doi.org/10.1109/TSMCC.2010.2050476

    Article  Google Scholar 

  33. Rahman A et al (2021) Multimodal EEG and Keystroke Dynamics Based Biometric System Using Machine Learning Algorithms. IEEE Access 9:94625–94643. https://doi.org/10.1109/ACCESS.2021.3092840

    Article  Google Scholar 

  34. Fox NA, Gross R, Cohn JF, Reilly RB (2007) Robust Biometric Person Identification Using Automatic Classifier Fusion of Speech, Mouth, and Face Experts. IEEE Trans Multimedia 9(4):701–714. https://doi.org/10.1109/TMM.2007.893339

    Article  Google Scholar 

  35. Xin Y et al (2018) Multimodal Feature-Level Fusion for Biometrics Identification System on IoMT Platform. IEEE Access 6:21418–21426. https://doi.org/10.1109/ACCESS.2018.2815540

    Article  Google Scholar 

  36. Toh KA, Yau WY (2004) "Combination of hyperbolic functions for multimodal biometrics data fusion. IEEE Trans Syst Man Cybern Part B 34(2):1196–1209. https://doi.org/10.1109/TSMCB.2003.821868

    Article  Google Scholar 

  37. Talreja V, Valenti MC, Nasrabadi NM (2021) Deep Hashing for Secure Multimodal Biometrics. IEEE Trans Inf Forensics Secur 16:1306–1321. https://doi.org/10.1109/TIFS.2020.3033189

    Article  Google Scholar 

  38. Li J, Hong D, Gao L, Yao J, Zheng K, Zhang B, Chanussot J (2022) Deep learning in multimodal remote sensing data fusion: A comprehensive review. Int J Appl Earth Obs Geoinformation (Vol. 112, p. 102926). Elsevier BV. https://doi.org/10.1016/j.jag.2022.102926

  39. Wang Y, Shi D, Zhou W (2022) Convolutional Neural Network Approach Based on Multimodal Biometric System with Fusion of Face and Finger Vein Features. Sensors (Basel) 22(16):6039. https://doi.org/10.3390/s22166039

    Article  Google Scholar 

  40. Gavrilova M, Luchak I, Sudhakar T, Tumpa SN (2022) Artificial Intelligence in Biometrics: Uncovering Intricacies of Human Body and Mind. In Learning and Analytics in Intelligent Systems (pp. 123–169). Springer International Publishing. https://doi.org/10.1007/978-3-030-93052-3_7

Download references

Funding

This research does not receive any kind of funding in any form.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mohammad Shabaz.

Ethics declarations

This research does not involve any animal or human participation.

Conflict of interest

The authors report no conflict of Interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Byeon, H., Raina, V., Sandhu, M. et al. Artificial intelligence-Enabled deep learning model for multimodal biometric fusion. Multimed Tools Appl 83, 80105–80128 (2024). https://doi.org/10.1007/s11042-024-18509-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-024-18509-0

Keywords

Navigation