Skip to main content
SpringerLink
Log in

Fusion of deep-learned and hand-crafted features for cancelable recognition systems

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The recent years have witnessed a dramatic shift in the way of biometric identification, authentication, and security processes. Among the essential challenges that face these processes are the online verification and authentication. These challenges lie in the complexity of such processes, the necessity of the personal real-time identifiable information, and the methodology to capture temporal information. In this paper, we present an integrated biometric recognition method to jointly recognize face, iris, palm print, fingerprint and ear biometrics. The proposed method is based on the integration of the extracted deep-learned features together with the hand-crafted ones by using a fusion network. Also, we propose a novel convolutional neural network (CNN)-based model for deep feature extraction. In addition, several techniques are exploited to extract the hand-crafted features such as histogram of oriented gradients (HOG), oriented rotated brief (ORB), local binary patterns (LBPs), scale-invariant feature transform (SIFT), and speeded-up robust features (SURF). Furthermore, for dimensional consistency between the combined features, the dimensions of the hand-crafted features are reduced using independent component analysis (ICA) or principal component analysis (PCA). The core of this paper is the template protection via a cancelable biometric scheme without significantly affecting the recognition performance. Specifically, we have used the bio-convolving approach to enhance the user’s privacy and ensure the robustness against spoof attacks. Additionally, various CNN hyper-parameters with their impact on the proposed model performance are studied. Our experiments on various datasets revealed that the proposed method achieves 96.69%, 95.59%, 97.34%, 96.11% and 99.22% recognition accuracies for face, iris, fingerprint, palm print and ear recognition, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  • Ali H, Jahangir U, Yousuf B, Noor A (2017) Human action recognition using SIFT and HOG method. In: Proceedings of the international conference on information and communication technologies (ICICT), pp 6–10

  • AMI ear database (2018), http://ctim.ulpgc.es/research_works/ami_ear_database/ Accessed Nov 2018

  • Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. In: IEEE transactions on pattern analysis and machine intelligence, pp 1798–1828

  • Beveridge J, Phillips J, Bolme D, Draper B, Givens G, Lui Y, Teli M, Zhang H, Scruggs W, Bowyer K (2013) The challenge of face recognition from digital point-and-shoot cameras. In: Proceedings of IEEE international conference biometrics, theory, application system, pp 1–8

  • Bolle RM, Connel JH, Ratha NK (2002) Biometrics perils and patches. In: Pattern Recognation, pp 2727–2738

  • CASIA Fingerprint Image Database Version 5.0, http://biometrics.idealtest.org/dbDetailForUser.do?id=7. Accessed Nov 2018

  • CASIA-IrisV3 Database (2018), http://www.cbsr.ia.ac.cn/english/IrisDatabase.asp. Accessed May 2018

  • COEP Palm Print Database (2018) http://www.coep.org.in/resources/coeppalmprintdatabase.m Accessed Dec 2018

  • Cheng C, Wang X, Li X (2017) UAV image matching based on surf feature and harris corner algorithm. In: Proceedings of the international conference on smart and sustainable city (ICSSC), pp 1–6

  • Chin C, Jin A, Ling D (2006) High security iris verification system based on random secret integration. In: Elsevier computer vision and image understanding, pp 169–177

  • Connie T, Jin T, Ong M, Ling D (2005) Ling D (2005) An automated palmprint recognition system. Image Vis Comput 23:501–515

    Article  Google Scholar 

  • Daugman J (2016) Information theory and the iris code. In: IEEE transactions on information forensics and security, pp 400–409

  • Deng W, Yao R, Zhao H, Yang X, Li G (2017a) A novel intelligent diagnosis method using optimal LS-SVM with improved PSO algorithm. Soft Comput 23:2445

    Article  Google Scholar 

  • Deng W, Zhao H, Zou L, Li G, Yang X, Wu D (2017b) A novel collaborative optimization algorithm in solving complex optimization problems. Soft Comput 21:4387–4398. https://doi.org/10.1007/s00500-016-2071-8

    Article  Google Scholar 

  • Ding C, Tao D (2017) Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach Intell 40(4):1002–1014

    Article  Google Scholar 

  • Ekinci M, Aykut M (2007) Gabor-based kernel PCA for palmprint recognition. Electr Lett 43:1077–1079

    Article  Google Scholar 

  • Emersic Z, Stepec D, Struc V (2017a) Training convolutional neural networks with limited training data for ear recognition in the wild. In: Automatic face gesture recognition, pp 987–994

  • Emersic Z, Struc V, Peer P (2017b) Ear recognition: more than a survey. Neurocomputing 255:26–39

    Article  Google Scholar 

  • Gangwar A, Joshi A (2016) Deepirisnet: deep iris representation with applications in iris recognition and cross-sensor iris recognition. In: IEEE international conference on image processing (ICIP), pp 2301–2305

  • Harjoko A, Hartati S, Dwiyasa H (2009) A method for iris recognition based on 1d coiflet wavelet. In: World academy of science engineering and technology, pp 126–129

  • Hasnat A, Bohn´e J, Milgram J, Gentric S, Chen L (2017) DeepVisage: making face recognition simple yet with powerful generalization Skills. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–12

  • Iandola F, Han S, Moskewicz M (2016) Squeezenet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. In: arXiv preprint arXiv:1602.07360

  • Ignat A, Luca M, Ciobanu A (2013) Iris features using dual tree complex wavelet transform in texture evaluation for biometrical identification. In: IEEE E-health and bioengineering conference, pp 1–4

  • Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the international conference on machine learning (ICML). arXiv:1502.03167v3

  • Jain AK (2004) An introduction to biometric recognition. In: IEEE transactions on circuits and systems for video technology, pp 4–20

  • Jain AK (2006) A tool for information Security. In: IEEE transactions on information forensics and security, pp 125–143

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  • Liu Y, Li H, Wang X (2017) Rethinking feature discrimination and polymerization for large-scale recognition. arXiv preprint arXiv:1710.00870

  • Liu N, Zhang M, Li H, Sun Z, Tan T (2016) Deepiris: learning pairwise filter bank for heterogeneous iris verification. Pattern Recognit Lett 82:154–161

    Article  Google Scholar 

  • Mariana-Iuliana G, Radu T, Marius P (2019) Local learning with deep and handcrafted features for facial expression recognition. arXiv:1804.10892v6

  • NIST (2012) IREX III—performance of iris identification algorithms. In: National Institute of Science and Technology, USA, Technical Report NIST Interagency Report 7836

  • Omara I, Li F, Zhang H, Zuo W (2016) A novel geometric feature extraction method for ear recognition. Expert Syst Appl 65:127–135

    Article  Google Scholar 

  • Patel V, Ratha N, Chellappa R (2015) Cancelable biometrics: a review. In: IEEE signal processing magazine, pp 54–65

  • Pedro L, Galdamez P, Raveane W (2016) A brief review of the ear recognition process using deep neural networks. J. Appl. Logic 24:62–70

    MathSciNet  Google Scholar 

  • Pei S, Chen M, Yu Y, Tang S, Zhong C (2017) Compact LBP and WLBP descriptor with magnitude and direction difference for face recognition. In: Proceedings of the IEEE international conference on image processing (ICIP), pp. 1067–1071

  • Phillips P, Scruggs W, O’Toole A, Flynn P, Bowyer K, Schott C, Sharpe M (2010) Frvt 2006 and ice 2006 large scale experimental results. In: IEEE transactions on pattern analysis and machine intelligence, pp 831–846

  • Pichao W, Zhaoyang L, Yonghong H, Wanqing L (2016) Combining convnets with hand-crafted features for action recognition based on an HMM-SVM Classifier. In: Proceedings of computer vision and pattern recognition (CVPR). arXiv:1602.00749

  • Pillai JK, Patel VM, Chellappa R, Ratha N (2010) Sectored random projections for cancelable iris biometrics. In: IEEE international conference on acoustics speech and signal processing, pp 1838–1841

  • Polash P, Gavrilova M, Klimenko S (2014) Situation awareness of cancelable biometric system. J Vis Comput 30:1059–1067

    Article  Google Scholar 

  • Ratha N, Connell J, Bolle R, Chikkerur S (2006) Cancelable biometrics: a case study in fingerprints. In: International conference of pattern recognition, pp 370–373

  • Ratha N, Chikkerur S, Connell J, Bolle R (2007) Generating cancelable fingerprint templates. IEEE Trans Pattern Anal Mach Intell 29:561–572

    Article  Google Scholar 

  • Rathgeb C (2010) A secure iris recognition based on local intensity variations. In: Proceedings of the 7th international conference on image analysis and recognition, pp 266–275

  • Rathgeb Ch, Barrero M, Busch Ch, Galbally J, Fierrez J (2015) Towards cancelable multi-biometrics based on bloom filters: a case study on feature level fusion of face and iris. In: 3rd International workshop on biometrics and forensics (IWBF)

  • Rathgeb C, Breitinger F, Busch C, Baier H (2014) On the application of bloom filters to iris biometrics. IET J Biom 3(4):207–218

    Article  Google Scholar 

  • Rathgeb C, Breitinger F, Baier H, Busch C (2015) Towards bloom filter-based indexing of iris biometric data. In: 15th IEEE international conference on biometrics, pp 422–429

  • Sarbadhikari S, Basak J, Pal S, Kundu M (1998) Noisy fingerprints classification with directional based features using MLP. Neural Comput Appl 7:180–191

    Article  Google Scholar 

  • Sh X, Jinhui T, Hanjiang L, Zhiheng N, Shuicheng Y (2016) Kinship Guided Age Progression. Pattern Recognit 59:156–157

    Article  Google Scholar 

  • Sun Y, Wang X, Tang X (2015) Deeply learned face representations are sparse, selective, and robust. In: IEEE conference on computer vision and pattern recognition

  • Swathi K, Kalyana V, Quek C (2018) Evolutionary based ICA with reference for EEGμRhythm Extraction. In: IEEE Access, pp 19702–19713

  • Sylvia W, Kamalaharidharini T (2017) Robust face recognition and classification system based on SIFT and DCP techniques in image processing. In: Proceedings of the international conference on intelligent computing and control (I2C2), pp 1–8

  • Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: Proceedings of IEEE CVPR, pp 1701–1708

  • Taigman Y, Yang M, Ranzato M, Wolf L (2015) Web-scale training for face identification. In: IEEE conference on computer vision and pattern recognition

  • Sun Y, Liang D, Wang X, Tang X (2015) DeepID3: Face recognition with very deep neural networks. arXiv:1502.00873

  • Tarek M, Ouda O, Hamza T (2017) Pre-image resistant cancelable biometrics scheme using bidirectional memory model. Int J Netw Secur 19(4):498–506

    Google Scholar 

  • Teoh ABJ, Ngo DCL, Goh A (2004) Biohashing: two factor authentication featuring fingerprint data and tokenised random number. Pattern Recognit 37(11):2245–2255

    Article  Google Scholar 

  • Theodoridis S, Koutroumbas K (2008) Pattern recognition, 4th edn. Academic Press, Cambridge

    MATH  Google Scholar 

  • Tomas M, Sule Y, Jon Y (2016) Combining deep learning and hand-crafted features for skin lesion classification. In: Sixth international conference on image processing theory, tools and applications (IPTA)

  • Vinay A, Kumar C, Shenoy R (2015) ORB-PCA based feature extraction technique for face recognition. In: Proceedings of the second international symposium on computer vision and the internet, pp 614–621

  • Xiaolin X, Yicong Z (2019) Two-dimensional quaternion PCA and sparse PCA. IEEE Trans Neural Netw Learn Syst 30(7):1–15. https://doi.org/10.1109/TNNLS.2018.2872541

    Article  Google Scholar 

  • Xu X, Lu L, Zhang X, Lu H, Deng W (2016) Multispectral palmprint recognition using multiclass projectionextreme learning machine and digital shearlet transform. Neural Comput Appl 27:143–153

    Article  Google Scholar 

  • Yakopcic C, Alom M, Taha T (2017) Extremely parallel memristor crossbar architecture for convolutionalneural network implementation. In: Proceedings of the international joint conference on neural networks (IJCNN), pp 1696–1703

  • Yang G, Shi D, Quek C (2005) Fingerprint minutia recognition with fuzzy neural network. In: Wang J, Liao X-F, Yi Z (eds) ISNN 2005, Part II. LNCS, vol 3497. Springer, Heidelberg, pp 165–170 (2005)

  • Yong-Xia L, Jin Q, Rui X (2010) A new detection method of singular points of fingerprints based on neural network. In: The 3rd IEEE international conference on computer science and information technology, ICCSIT

  • Yuan Z, Liu Y, Yue J, Li J, Yang H (2017) CORAL: coarse-grained reconfigurable architecture for convolutional neural networks. In: Proceedings of the IEEE/ACM international symposium on low power electronics and design (ISLPED), pp 1–6

  • Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Zhao H, Sun M, Deng W, Yang X (2017) A new feature extraction method based on EEMD and multi-scale fuzzy entropy for motor bearing. Entropy 19:14

    Article  Google Scholar 

  • Zhao H, Yao R, Xu L, Yuan Y, Li G, Deng W (2018) Study on a novel fault damage degree identification method using high-order differential mathematical morphology gradient spectrum entropy. Entropy 20:682

    Article  Google Scholar 

  • Zheng Y, Pal D, Savvides M (2018) Ring loss: convex feature normalization for face recognition. In: CVPR

  • Zhu E, Yin J, Hu C, Zhang G (2005) Quality estimation of fingerprint image based on neural network. In: Wang L, Chen KS, Ong Y (eds.) ICNC 2005, Part II. LNCS, vol. 3611, pp. 65–70. Springer, Heidelberg

  • Zuo J, Ratha NK, Connel JH (2008) Cancelable iris biometric. In: Proceedings of the 19th international conference on pattern recognition, pp 1–4

Download references

Author information

Authors and Affiliations

  1. Electronics and Communication Department, Delta Higher Institute for Engineering and Technology (DHIET), Mansoura, Egypt

    Essam Abdellatef

  2. Department of Nuclear Safety and Radiological Emergencies, NCRRT, Egyptian Atomic Energy Authority (EAEA), Cairo, Egypt

    Eman M. Omran & Ayman A. Eisa

  3. Mathematics and Computer Science Department, Faculty of Science, Menoufia University, Menofia Governorate, 32511, Egypt

    Randa F. Soliman

  4. Department of Computer Science and Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Nabil A. Ismail & Salah Eldin S. E. Abd Elrahman

  5. Department of Computer Science, Durham University, Durham, DH1 3LE, UK

    Khalid N. Ismail

  6. Information Technology Department, Faculty of Computers and Information, Menoufia University, Al Minufiyah, Egypt

    Khalid N. Ismail

  7. Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Mohamed Rihan & Fathi E. Abd El-Samie

  8. Department of Information Technology, College of Computer and Information sciences, Princess Nourah Bint Abdulrahman University, Riyadh, Saudi Arabia

    Fathi E. Abd El-Samie

Authors
  1. Essam Abdellatef
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eman M. Omran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Randa F. Soliman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nabil A. Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Salah Eldin S. E. Abd Elrahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Khalid N. Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mohamed Rihan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Fathi E. Abd El-Samie
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ayman A. Eisa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Essam Abdellatef.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abdellatef, E., Omran, E.M., Soliman, R.F. et al. Fusion of deep-learned and hand-crafted features for cancelable recognition systems. Soft Comput 24, 15189–15208 (2020). https://doi.org/10.1007/s00500-020-04856-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-04856-1

Keywords

Search

Navigation