Skip to main content

Advertisement

Springer Nature Link
Log in

A comprehensive survey and deep learning-based approach for human recognition using ear biometric

  • Survey
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Human recognition systems based on biometrics are much in demand due to increasing concerns of security and privacy. The human ear is unique and useful for recognition. It offers numerous advantages over popular biometrics traits face, iris, and fingerprints. A lot of work has been attributed to ear biometric, and the existing methods have achieved remarkable success over constrained databases. However, in unconstrained environment, a significant level of difficulty is observed as the images experience various challenges. In this paper, we first have provided a comprehensive survey on ear biometric using a novel taxonomy. The survey includes in-depth details of databases, performance evaluation parameters, and existing approaches. We have introduced a new database, NITJEW, for evaluation of unconstrained ear detection and recognition. A modified deep learning models Faster-RCNN and VGG-19 are used for ear detection and ear recognition tasks, respectively. The benchmark comparative assessment of our database is performed with six existing popular databases. Lastly, we have provided insight into open-ended research problems worth examining in the near future. We hope that our work will be a stepping stone for new researchers in ear biometrics and helpful for further development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Anand, R., Shanthi, T., Nithish, M., Lakshman, S.: Face recognition and classification using googlenet architecture. In: Soft Computing for Problem Solving, pp. 261–269. Springer, Berlin (2020)

    Chapter  Google Scholar 

  2. Liu, Y., Zhou, B., Han, C., Guo, T., Qin, J.: A novel method based on deep learning for aligned fingerprints matching. Appl. Intell. 50(2), 397–416 (2020)

    Article  Google Scholar 

  3. Thakkar, S., Patel, C.: Iris recognition supported best gabor filters and deep learning cnn options. In: 2020 International Conference on Industry 4.0 Technology (I4Tech), pp. 167–170. IEEE (2020)

  4. Zhao, S., Zhang, B.: Deep discriminative representation for generic palmprint recognition. Pattern Recogn. 98, 107071 (2020)

    Article  Google Scholar 

  5. Trabelsi, S., Samai, D., Meraoumia, A., Bensid, K., Benlamoudi, A., Dornaika, F., Taleb-Ahmed, A.: Finger-knuckle-print recognition using deep convolutional neural network. In: 020 1st International Conference on Communications, Control Systems and Signal Processing (CCSSP), (pp. 163–168). IEEE (2020)

  6. Alshazly, H., Linse, C., Barth, E., Martinetz, T.: Deep convolutional neural networks for unconstrained ear recognition. IEEE Access 8, 170295–170310 (2020)

    Article  Google Scholar 

  7. Sabhanayagam, T., Venkatesan, V.P., Senthamaraikannan, K.: A comprehensive survey on various biometric systems. Int. J. Appl. Eng. Res. 13(5), 2276–2297 (2018)

    Google Scholar 

  8. Chauhan, S., Arora, A., Kaul, A.: A survey of emerging biometric modalities. Procedia Computer Science, 2:213 – 218. Proceedings of the International Conference and Exhibition on Biometrics Technology (2010)

  9. Vats, S., Harkeerat Kaur, G.: A comparative study of different biometric features. International Journal of Advanced Research in Computer Science 7(6), (2017)

  10. Alsaadi, I.: Physiological biometric authentication systems, advantages, disadvantages and future development: A review. Int. J. Sci. Technol. Res. 4, 285–289 (2015)

    Google Scholar 

  11. Bertillon, A.: La photographie judiciaire: avec un appendice sur la classification et l’identification anthropométriques. Gauthier-Villars, Paris (1890)

    Google Scholar 

  12. Iannarelli, A.: Ear identification. Paramount Publishing Company, Forensic Identification Series (1989)

  13. Van der Lugt, C.: Ear prints (2000)

  14. Kasprzak, J.: Forensic otoscopy-new method of human identification (2015)

  15. Ibrahim, M.I.S., Nixon, M.S., Mahmoodi, S.: The effect of time on ear biometrics. In: 2011 International Joint Conference on Biometrics (IJCB), 1–6 (2011)

  16. Bowyer, K.W., Sarkar, S., Victor, B.: Comparison and combination of ear and face images in appearance-based biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1160–1165 (2003)

    Article  Google Scholar 

  17. Uttara Athawale, M.G.: Survey on recent ear biometric recognition techniques. Int. J. Comput. Sci. Eng. 6, 1208–1211 (2018)

    Google Scholar 

  18. Abaza, A., Bourlai, T.: On ear-based human identification in the mid-wave infrared spectrum. Image Vis. Comput. 31(9), 640–648 (2013)

    Article  Google Scholar 

  19. Liu, Y., Lu, Z., Li, J., Yang, T., Yao, C.: Global temporal representation based cnns for infrared action recognition. IEEE Signal Process. Lett. 25(6), 848–852 (2018)

    Article  Google Scholar 

  20. Liu, Y., Lu, Z., Li, J., Yao, C., Deng, Y.: Transferable feature representation for visible-to-infrared cross-dataset human action recognition. Complexity, 2018 (2018)

  21. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., and Garnett, R., editors, Advances in Neural Information Processing Systems 28, pp. 91–99. Curran Associates, Inc (2015)

  22. Liu, S., Deng, W.: Very deep convolutional neural network based image classification using small training sample size. In: 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 730–734 (2015)

  23. Kumar, A., Wu, C.: Automated human identification using ear imaging. Pattern Recogn. 45(3), 956–968 (2006)

    Article  Google Scholar 

  24. Prakash, S., Gupta, P.: An efficient ear localization technique. Image Vis. Comput. 30(1), 38–50 (2012)

    Article  Google Scholar 

  25. USTB (2004).Ear Recoginition Laboratory( University of science and technologyBeijing USTB database). Retrieved fromhttp://www1.ustb.edu.cn/resb/en/doc/Imagedb_123_intro_en.pdf

  26. Yan, P., Bowyer, K.W.: Biometric recognition using three dimensional ear shape cvrl data sets ( university of notre dame und database). retrieved from https://sites.google.com/a/nd.edu/public-cvrl/data-sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(8):1297–1308 (2003)

  27. UMIST (2014). Face database. ( university of sheffield). Available at: http://www.shef.ac.uk/eee/research/vie/research/face.html

  28. Gonzalez, E.: Ph.d. thesis, ami datbase. http://ctim.ulpgc.es/researchworks/amieardatabase/ (2008)

  29. Emersic, Z., Struc, V., Peer, P.: Ear recognition: More than a survey. Neurocomputing 255, 26–39 (2017)

    Article  Google Scholar 

  30. Raposo, R., Hoyle, E., Peixinho, A., Proença, H.: Ubear: A dataset of ear images captured on-the-move in uncontrolled conditions. In: 2011 IEEE Workshop on Computational Intelligence in Biometrics and Identity Management (CIBIM), pp. 84–90 (2011)

  31. Zhou, Y., Zaferiou, S.: Deformable models of ears in-the-wild for alignment and recognition. In: 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017), pp. 626–633 (2017)

  32. Zhang, Y., Mu, Z., Yuan, L., Yu, C., Liu, Q.: Ustb-helloear: A large database of ear images photographed under uncontrolled conditions. In: Zhao, Y., Kong, X., Taubman, D. (eds.) Image and Graphics, pp. 405–416. Springer International Publishing, Cham (2017)

    Chapter  Google Scholar 

  33. Hoang, V.T.: Earvn1.0: A new large-scale ear images dataset in the wild. Data in Brief, 27: 104630 (2019)

  34. Awad, A.I., Hassaballah, M.: Image feature detectors and descriptors. Studies in Computational Intelligence. Springer International Publishing, Cham (2016)

    Book  Google Scholar 

  35. Prakash, S., Jayaraman, U., Gupta, P.: Ear localization using hierarchical clustering. In: Optics and Photonics in Global Homeland Security V and Biometric Technology for Human Identification VI, volume 7306, pp. 730620. International Society for Optics and Photonics (2009)

  36. Abaza, A., Hebert, C., Harrison, M. A. F.: Fast learning ear detection for real-time surveillance. In: 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), 1–6 (2010)

  37. Joshi, K.V., Chauhan, N.C.: Edge detection and template matching approaches for human ear detection. IJCA Special Issue on Intelligent Systems and Data Processing 50–55 (2011)

  38. Wahab, N. K. A., Hemayed, E. E., Fayek, M. B.: Heard: An automatic human ear detection technique. In: 2012 International Conference on Engineering and Technology (ICET), (pp. 1–7) (2012)

  39. Ganesh, M.R., Krishna, R., Manikantan, K., Ramachandran, S.: Entropy based binary particle swarm optimization and classification for ear detection. Eng. Appl. Artif. Intell. 27, 115–128 (2014)

    Article  Google Scholar 

  40. Chidananda, P., Srinivas, P., Manikantan, K., Ramachandran, S.: Entropy-cum-hough-transform-based ear detection using ellipsoid particle swarm optimization. Mach. Vis. Appl. 26(2), 185–203 (2015)

    Article  Google Scholar 

  41. Pflug, A., Winterstein, A., Busch, C.: Robust localization of ears by feature level fusion and context information. In: 2013 International Conference on Biometrics (ICB), pp. 1–8 (2013)

  42. Halawani, A., Li, H.: Human ear localization: A template-based approach. Int. J. Signal Process. Sys. 4(3), 258–262 (2016)

    Article  Google Scholar 

  43. Resmi, K. R., Raju, G.: A novel approach to automatic ear detection using banana wavelets and circular hough transform. In: 2019 International Conference on Data Science and Communication (IconDSC), pp. 1–5 (2019)

  44. Marsico, M. D., Michele, N., Riccio, D.: Hero: Human ear recognition against occlusions. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 178–183 (2010)

  45. Bustard, J.D., Nixon, M.S.: Toward unconstrained ear recognition from two-dimensional images. IEEE Trans. Sys. Man. Cybern-Part A: Sys. Humans 40(3), 486–494 (2010)

    Article  Google Scholar 

  46. Arbab-Zavar, B., Nixon, M.S.: On guided model-based analysis for ear biometrics. Comput. Vis. Image Underst. 115(4), 487–502 (2011)

    Article  Google Scholar 

  47. Kumar, A., Wu, C.: Automated human identification using ear imaging. Pattern Recogn. 45(3), 956–968 (2012)

    Article  Google Scholar 

  48. Chan, T. S., Kumar, A.: Reliable ear identification using 2-d quadrature filters. Pattern Recognition Letters, Novel Pattern Recognition-Based Methods for Re-identification in Biometric Context (2012) 33(14):1870 – 1881

  49. Sibai, F.N., Nuaimi, A., Maamari, A., Kuwair, R.: Ear recognition with feed-forward artificial neural networks. Neural Comput. Appl. 23(5), 1265–1273 (2013)

    Article  Google Scholar 

  50. Boodoo-Jahangeer, N. B., Baichoo, S.: Lbp-based ear recognition. In: 13th IEEE International Conference on BioInformatics and BioEngineering, 1–4 (2013)

  51. Pflug, A., Busch, C., Ross, A.: 2d ear classification based on unsupervised clustering. IEEE International Joint Conference on Biometrics 1–8 (2014)

  52. Yuan, L., Mu, Z.: Ear recognition based on gabor features and kfda. The Scientific World Journal (2014)

  53. Nigam, A., Gupta, P.: Robust ear recognition using gradient ordinal relationship pattern. In: Computer Vision - ACCV 2014, volume 9010 of Lecture Notes in Computer Science, (pp. 617–632). Springer, Berlin (2014)

  54. Anwa, A.S., Kamal, K., Ghany, A., Elmahdy, H.: Human ear recognition using geometrical features extraction. Procedia Computer Science, 65(Supplement C):529 – 537. International Conference on Communications, management, and Information technology (ICCMIT’2015) (2015)

  55. Emersic, Z., Peer, P.: Toolbox for ear biometric recognition evaluation. In: IEEE EUROCON 2015 - International Conference on Computer as a Tool (EUROCON), 1–6 (2015)

  56. Hassaballah, M., Alshazly, H.A., Ali, A.A.: Ear recognition using local binary patterns: A comparative experimental study. Expert Syst. Appl. 118, 182–200 (2019)

    Article  Google Scholar 

  57. Birajadar, P., Haria, M., Sangodkar, S. G., Gadre, V.: Unconstrained ear recognition using deep scattering wavelet network. In: 2019 IEEE Bombay Section Signature Conference (IBSSC), 1–6. IEEE (2019)

  58. Hassaballah, M., Alshazly, H., Ali, A.A.: Robust local oriented patterns for ear recognition. Multimedia Tools Appl. 79(41), 31183–204 (2020)

    Article  Google Scholar 

  59. Sajadi, S., Fathi, A.: Genetic algorithm based local and global spectral features extraction for ear recognition. Expert Syst. Appl. 159, 113639 (2020)

    Article  Google Scholar 

  60. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 2278–2324 (1998)

  61. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)

  62. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: Ssd: Single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision - ECCV, pp. 21–37. Springer International Publishing, Cham (2016)

    Google Scholar 

  63. Najibi, M., Samangouei, P., Chellappa, R., Davis, L. S.: Ssh: Single stage headless face detector. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp 4885–4894 (2017)

  64. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778 (2016)

  65. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. Proceedings of the IEEE conference on computer vision and pattern recognition 815–823 (2015)

  66. Zhang, Y., Mu, Z.: Ear detection under uncontrolled conditions with multiple scale faster region-based convolutional neural networks. Symmetry 9(4), 23 (2017)

    Article  MathSciNet  Google Scholar 

  67. Cintas, C., Quinto-Sánchez, M., Acuña, V., Paschetta, C., de Azevedo, S., de Cerqueira, C.C.S., Ramallo, V., Gallo, C., Poletti, G., Bortolini, M.C., Canizales-Quinteros, S., Rothhammer, F., Bedoya, G., Ruiz-Linares, A., Gonzalez-José, R., Delrieux, C.: Automatic ear detection and feature extraction using geometric morphometrics and convolutional neural networks. IET Biometrics 6(3), 211–223 (2017)

    Article  Google Scholar 

  68. Emersic, Z., Gabriel, L.L., Struc, V., Peer, P.: Convolutional encoder-decoder networks for pixel-wise ear detection and segmentation. IET Biometrics 7(3), 175–184 (2018)

    Article  Google Scholar 

  69. Raveane, W., Galdámez, P.L., González Arrieta, M.A.: Ear detection and localization with convolutional neural networks in natural images and videos. Processes 7(7), 457 (2019)

    Article  Google Scholar 

  70. Kamboj, A., Rani, R., Nigam, A., Jha, R.R.: Ced-net: context-aware ear detection network for unconstrained images. Pattern Analysis and Applications 1–22 (2020)

  71. Tian, L., Mu, Z.: Ear recognition based on deep convolutional network. In: 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 437–441 (2016)

  72. Hansley, E.E., Segundo, M.P., Sarkar, S.: Employing fusion of learned and handcrafted features for unconstrained ear recognition. IET Biometrics 7(3), 215–223 (2018)

    Article  Google Scholar 

  73. Emersic, Z., Stepec, D., Struc, V., Peer, P.: Training convolutional neural networks with limited training data for ear recognition in the wild. In: 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017), pp. 987–994 (2017)

  74. Zhang, Y., Mu, Z., Yuan, L., Yu, C.: Ear verification under uncontrolled conditions with convolutional neural networks. IET Biometrics 7(3), 185–198 (2018)

    Article  Google Scholar 

  75. Emeršič, Ž, Križaj, J., Štruc, V., Peer, P.: Deep Ear Recognition Pipeline, pp. 333–362. Springer International Publishing, Cham (2019)

    Google Scholar 

  76. Priyadharshini, R.A., Arivazhagan, S., Arun, M.: A deep learning approach for person identification using ear biometrics. Applied Intelligence 1–12 (2020)

  77. Štepec, D., Emeršič, Ž., Peer, P., Štruc, V.: Constellation-Based Deep Ear Recognition (pp. 161–190). Springer International Publishing: Cham (2020)

  78. Radhika, K., Devika, K., Aswathi, T., Sreevidya, P., Sowmya, V., Soman, K.: Performance analysis of nasnet on unconstrained ear recognition. In: Nature Inspired Computing for Data Science (pp. 57–82). Springer, Berlin (2020)

  79. Ganapathi, I.I., Prakash, S., Dave, I.R., Bakshi, S.: Unconstrained ear detection using ensemble-based convolutional neural network model. Concurr. Comput: Pract. Experience 32(1), e5197 (2020)

    Article  Google Scholar 

  80. Alshazly, H., Linse, C., Barth, E., Martinetz, T.: Handcrafted versus cnn features for ear recognition. Symmetry 11(12), 1493 (2019)

    Article  Google Scholar 

  81. Mustafa, A.S., Abdulelah, A.J., Ahmed, K.A.: Multimodal biometric system iris and fingerprint recognition based on fusion technique. Int. J. Adv. Sci. Technol. 29(3), 7423–7432 (2020)

    Google Scholar 

  82. Snelick, R., Uludag, U., Mink, A., Indovina, M., Jain, A.: Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 450–455 (2005)

    Article  Google Scholar 

  83. Jaswal, G., Kaul, A., Nath, R.: Knuckle print biometrics and fusion schemes-overview, challenges, and solutions. ACM Comput. Surv. (CSUR) 49(2), 1–46 (2016)

    Article  Google Scholar 

  84. Ghoualmi, L., Chikhi, S., Draa, A.: A SIFT-Based Feature Level Fusion of Iris and Ear Biometrics, pp. 102–112. Springer International Publishing, Cham (2015)

    Google Scholar 

  85. Rathore, R., Prakash, S., Gupta, P.: Efficient human recognition system using ear and profile face. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–6 (2013)

  86. Amirthalingam, G., Radhamani, G.: New chaff point based fuzzy vault for multimodal biometric cryptosystem using particle swarm optimization. J King Saud Univ - Comput Inform Sci 28(4), 381–394 (2016)

    Google Scholar 

  87. Hezil, N., Boukrouche, A.: Multimodal biometric recognition using human ear and palmprint. IET Biometrics 6(5), 351–359 (2017)

    Article  Google Scholar 

  88. Kumar, A. M., Chandralekha, A., Himaja, Y., Sai, S.M.: Local binary pattern based multimodal biometric recognition using ear and fkp with feature level fusion. In: 2019 IEEE International Conference on Intelligent Techniques in Control, Optimization and Signal Processing (INCOS), pp.1–5. IEEE (2019)

  89. Zibran, M.F.: Biometric authentication: The security issues. University of Saskatchewan (2012)

  90. Galbally, J., Marcel, S., Fierrez, J.: Biometric antispoofing methods: A survey in face recognition. IEEE Access 2, 1530–1552 (2014b)

    Article  Google Scholar 

  91. Marcel, S., Nixon, M.S., Li, S.Z.: Handbook of biometric anti-spoofing, vol. 1. Springer, Berlin (2014)

    Book  Google Scholar 

  92. Galbally, J., Fierrez, J., Ortega-Garcia, J., Cappelli, R.: Fingerprint anti-spoofing in biometric systems. In: Handbook of Biometric Anti-Spoofing pp. 35–64. Springer, Berlin (2014a)

  93. Sun, Z.,Tan, T.: Iris anti-spoofing. In: Handbook of biometric anti-spoofing (pp. 103–123). Springer: Berlin (2014)

  94. Nourmohammadi-Khiarak, J., Pacut, A.: An ear anti-spoofing database with various attacks. In: 2018 International Carnahan Conference on Security Technology (ICCST), (pp. 1–5). IEEE (2018)

  95. Toprak, İ, Toygar, Ö.: Ear anti-spoofing against print attacks using three-level fusion of image quality measures. SIViP 14(2), 417–424 (2020)

    Article  Google Scholar 

  96. Sepas-Moghaddam, A., Pereira, F., Correia, P. L.: Ear presentation attack detection: Benchmarking study with first lenslet light field database. In: 2018 26th European Signal Processing Conference (EUSIPCO), pages 2355–2359. IEEE (2018)

  97. Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: Labelme: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77(1–3), 157–173 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Institute of Technology Jalandhar, Jalandhar, Punjab, 144011, India

    Aman Kamboj & Rajneesh Rani

  2. Indian Institute of Technology Mandi, Mandi, Himachal Pradesh, 175005, India

    Aditya Nigam

Authors
  1. Aman Kamboj
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Rajneesh Rani
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Aditya Nigam
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Aman Kamboj.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kamboj, A., Rani, R. & Nigam, A. A comprehensive survey and deep learning-based approach for human recognition using ear biometric. Vis Comput 38, 2383–2416 (2022). https://doi.org/10.1007/s00371-021-02119-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02119-0

Keywords