Skip to main content
SpringerLink
Log in
Book cover

Recent Advances in Computer Vision pp 333–362Cite as

Deep Ear Recognition Pipeline

  • Chapter
  • First Online:

Part of the book series: Studies in Computational Intelligence ((SCI,volume 804))

Abstract

Ear recognition has seen multiple improvements in recent years and still remains very active today. However, it has been approached from recognition and detection perspective separately. Furthermore, deep-learning-based approaches that are popular in other domains have seen limited use in ear recognition and even more so in ear detection. Moreover, to obtain a usable recognition system a unified pipeline is needed. The input in such system should be plain images of subjects and the output identities based only on ear biometrics. We conduct separate analysis through detection and identification experiments on the challenging dataset and, using the best approaches, present a novel, unified pipeline. The pipeline is based on convolutional neural networks (CNN) and presents, to the best of our knowledge, the first CNN-based ear recognition pipeline. The pipeline incorporates both, the detection of ears on arbitrary images of people, as well as recognition on these segmented ear regions. The experiments show that the presented system is a state-of-the-art system and, thus, a good foundation for future real-word ear recognition systems.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Abaza, A., Hebert, C., Harrison, M.A.F.: Fast learning ear detection for real-time surveillance. In: International Conference on Biometrics: Theory Applications and Systems, pp. 1–6. IEEE (2010)

    Google Scholar 

  2. Abaza, A., Ross, A., Hebert, C., Harrison, M.A.F., Nixon, M.: A survey on ear biometrics. ACM Comput. Surv. 45(2), 1–22 (2013)

    Article  Google Scholar 

  3. Alaraj, M., Hou, J., Fukami, T.: A neural network based human identification framework using ear images. In: International Technical Conference of IEEE Region, vol. 10, pp. 1595–1600. IEEE (2010)

    Google Scholar 

  4. Ansari, S., Gupta, P.: Localization of ear using outer helix curve of the ear. In: International Conference on Computing: Theory and Applications, pp. 688–692. IEEE (2007)

    Google Scholar 

  5. Arbab-Zavar, B., Nixon, M.S.: On shape-mediated enrolment in ear biometrics. In: International Symposium on Visual Computing, pp. 549–558. Springer (2007)

    Google Scholar 

  6. Arbab-Zavar, B., Nixon, M.S.: Robust log-Gabor filter for ear biometrics. In: International Conference on Pattern Recognition, pp. 1–4. IEEE (2008)

    Google Scholar 

  7. Attarchi, S., Faez, K., Rafiei, A.: A new segmentation approach for ear recognition. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 1030–1037. Springer (2008)

    Google Scholar 

  8. Badrinarayanan, V., Handa, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. arXiv:1505.07293 (2015)

  9. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  10. Banerjee, S., Chatterjee, A.: Robust multimodal multivariate ear recognition using kernel based simultaneous sparse representation. Eng. Appl. Artif. Intell. 64, 340–351 (2017)

    Article  Google Scholar 

  11. Baoqing, Z., Zhichun, M., Chen, J., Jiyuan, D.: A robust algorithm for ear recognition under partial occlusion. In: Chinese Control Conference, pp. 3800–3804 (2013)

    Google Scholar 

  12. Basit, A., Shoaib, M.: A human ear recognition method using nonlinear curvelet feature subspace. Int. J. Comput. Math. 91(3), 616–624 (2014)

    Article  Google Scholar 

  13. Benzaoui, A., Kheider, A., Boukrouche, A.: Ear description and recognition using ELBP and wavelets. In: International Conference on Applied Research in Computer Science and Engineering, pp. 1–6 (2015)

    Google Scholar 

  14. Benzaoui, A., Hezil, N., Boukrouche, A.: Identity recognition based on the external shape of the human ear. In: International Conference on Applied Research in Computer Science and Engineering, pp. 1–5. IEEE (2015)

    Google Scholar 

  15. Bourouba, H., Doghmane, H., Benzaoui, A., Boukrouche, A.H.: Ear recognition based on multi-bags-of-features histogram. In: International Conference on Control, Engineering Information Technology, pp. 1–6 (2015)

    Google Scholar 

  16. Bustard, J.D., Nixon, M.S.: Toward unconstrained ear recognition from two-dimensional images. Trans. Syst. Man Cybern. Part A: Syst. Hum. 40(3), 486–494 (2010)

    Article  Google Scholar 

  17. Carreira-Perpinan, M.A.: Compression neural networks for feature extraction: application to human recognition from ear images. Master’s thesis, Faculty of Informatics, Technical University of Madrid, Spain (1995)

    Google Scholar 

  18. Chan, T.S., Kumar, A.: Reliable ear identification using 2-D quadrature filters. Pattern Recogn. Lett. 33(14), 1870–1881 (2012)

    Article  Google Scholar 

  19. Chidananda, P., Srinivas, P., Manikantan, K., Ramachandran, S.: Entropy-cum-Hough-transform-based ear detection using ellipsoid particle swarm optimization. Mach. Vis. Appl. 26(2), 185–203 (2015)

    Article  Google Scholar 

  20. Chowdhury, D.P., Bakshi, S., Guo, G., Sa, P.K.: On applicability of tunable filter bank based feature for ear biometrics: a study from constrained to unconstrained. J. Med. Syst. 42(1), 11 (2018)

    Article  Google Scholar 

  21. Cummings, A.H., Nixon, M.S., Carter, J.N.: A novel ray analogy for enrolment of ear biometrics. In: International Conference on Biometrics: Theory Applications and Systems, pp. 1–6. IEEE (2010)

    Google Scholar 

  22. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on Computer Vision and Patten Recognition, pp. 886–893. IEEE (2005)

    Google Scholar 

  23. Damar, N., Fuhrer, B.: Ear recognition using multi-scale histogram of oriented gradients. In: Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 21–24 (2012)

    Google Scholar 

  24. Dewi, K., Yahagi, T.: Ear photo recognition using scale invariant keypoints. In: Computational Intelligence, pp. 253–258 (2006)

    Google Scholar 

  25. Dodge, S., Mounsef, J., Karam, L.: Unconstrained ear recognition using deep neural networks. IET Biom. (2018)

    Google Scholar 

  26. Dogucan, Y., Fevziye, E., Ekenel, H.: Domain adaptation for ear recognition using deep convolutional neural networks. IET Biom. 7(3), 199–206 (2018)

    Article  Google Scholar 

  27. Ear Recognition Laboratory at the University of Science & Technology Beijing: Introduction to USTB Ear Image Databases (2002). http://www1.ustb.edu.cn/resb/en/index.htm. Accessed 15 Mar 2018

  28. Earnest, H., Segundo, P., Sarkar, S.: Employing fusion of learned and handcrafted features for unconstrained ear recognition. IET Biom. 7(3), 215–223 (2018)

    Article  Google Scholar 

  29. Emeršič, Ž., Gabriel, L.L., Štruc, V., Peer, P.: Convolutional encoder-decoder networks for pixel-wise ear detection and segmentation. IET Biom. 7(3), 175–184 (2018)

    Article  Google Scholar 

  30. Emeršič, Ž., Meden, B., Peer, P., Štruc, V.: Evaluation and analysis of ear recognition models: performance, complexity and resource requirements. Neural Comput. Appl. 1–16

    Google Scholar 

  31. Emeršič, Ž., Meden, B., Peer, P., Štruc, V.: Covariate analysis of descriptor-based ear recognition techniques. In: 2017 International Conference and Workshop on Bioinspired Intelligence (IWOBI), pp. 1–9. IEEE (2017)

    Google Scholar 

  32. Emeršič, Ž., Meden, B., Peer, P., Štruc, V.: Evaluation and analysis of ear recognition models: performance, complexity and resource requirements. Neural Comput. Appl. 1–16 (2018)

    Google Scholar 

  33. Emeršič, Ž., Peer, P.: Ear biometric database in the wild. In: 2015 4th International Work Conference on Bioinspired Intelligence (IWOBI), pp. 27–32. IEEE (2015)

    Google Scholar 

  34. Emeršič, Ž., Štepec, D., Štruc, V., Peer, P.: Training convolutional neural networks with limited training data for ear recognition in the wild. In: 12th IEEE International Conference on Automatic Face and Gesture (FG 2017) (2017)

    Google Scholar 

  35. Emeršič, Ž., Štepec, D., Štruc, V., Peer, P., George, A., Ahmad, A., Omar, E., Boult, T.E., Safdari, R., Zhou, Y., Zafeiriou, S., Yaman, D., Eyiokur, F.I., Ekenel, H.K.: The unconstrained ear recognition challenge. In: International Joint Conference on Biometrics (IJCB) (2017)

    Google Scholar 

  36. Emeršič, Ž., Štruc, V., Peer, P.: Ear recognition: more than a survey. Neurocomputing 255, 26–39 (2017)

    Article  Google Scholar 

  37. Ganesh, M.R., Krishna, R., Manikantan, K., Ramachandran, S.: Entropy based binary particle swarm optimization and classification for ear detection. Eng. Appl. Artif. Intell. 27, 115–128 (2014)

    Article  Google Scholar 

  38. Gourier, N., Hall, D., Crowley, J.L.: Estimating face orientation from robust detection of salient facial structures. In: FG Net Workshop on Visual Observation of Deictic Gestures, vol. 6 (2004)

    Google Scholar 

  39. Guo, Y., Xu, Z.: Ear recognition using a new local matching approach. In: International Conference on Image Processing, pp. 289–292. IEEE (2008)

    Google Scholar 

  40. Han, J., Zhang, D., Cheng, G., Liu, N., Xu, D.: Advanced deep-learning techniques for salient and category-specific object detection: a survey. IEEE Signal Process. Mag. 35(1), 84–100 (2018)

    Article  Google Scholar 

  41. Hansley, E.E., Segundo, M.P., Sarkar, S.: Employing fusion of learned and handcrafted features for unconstrained ear recognition. arXiv:1710.07662 (2017)

  42. He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: European Conference on Computer Vision, pp. 630–645. Springer (2016)

    Google Scholar 

  43. He, X., Yu, Z., Wang, T., Lei, B.: Skin lesion segmentation via deep RefineNet. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 303–311. Springer (2017)

    Google Scholar 

  44. He, X., Yu, Z., Wang, T., Lei, B., Shi, Y.: Dense deconvolution net: multi path fusion and dense deconvolution for high resolution skin lesion segmentation. Technol. Health Care 1–10 (2018)

    Google Scholar 

  45. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 (2017)

  46. Islam, S.M., Bennamoun, M., Davies, R.: Fast and fully automatic ear detection using cascaded AdaBoost. In: Workshop on Applications of Computer Vision, pp. 1–6. IEEE (2008)

    Google Scholar 

  47. Jacobs, R.A.: Increased rates of convergence through learning rate adaptation. Neural Netw. 1(4), 295–307 (1988)

    Article  Google Scholar 

  48. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia, pp. 675–678. ACM (2014)

    Google Scholar 

  49. Kannala, J., Rahtu, E.: BSIF: Binarized statistical image features. In: International Conference on Pattern Recognition, pp. 1363–1366. IEEE (2012)

    Google Scholar 

  50. Križaj, J., Štruc, V., Pavešic, N.: Adaptation of SIFT features for robust face recognition. In: Image Analysis and Recognition, pp. 394–404. Springer (2010)

    Google Scholar 

  51. Kumar, A., Wu, C.: Automated human identification using ear imaging. Pattern Recogn. 45(3), 956–968 (2012)

    Article  Google Scholar 

  52. Lin, G., Milan, A., Shen, C., Reid, I.: RefineNet: Multi-path refinement networks for high-resolution semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 5 (2017)

    Google Scholar 

  53. Lu, Z., Jiang, X., Kot, A.: Deep coupled resnet for low-resolution face recognition. Signal Process. Lett. 25(4), 526–530 (2018)

    Article  Google Scholar 

  54. Meraoumia, A., Chitroub, S., Bouridane, A.: An automated ear identification system using Gabor filter responses. In: International Conference on New Circuits and Systems, pp. 1–4. IEEE (2015)

    Google Scholar 

  55. Messer, K., Matas, J., Kittler, J., Luettin, J., Maitre, G.: XM2VTSDB: the extended M2VTS database. In: International Conference on Audio and Video-Based Biometric Person Authentication, vol. 964, pp. 965–966 (1999)

    Google Scholar 

  56. Moody, J., Hanson, S., Krogh, A., Hertz, J.A.: A simple weight decay can improve generalization. Adv. Neural Inf. Process. Syst. 4, 950–957 (1995)

    Google Scholar 

  57. University of Notre Dame: Face database (2015). https://sites.google.com/a/nd.edu/public-cvrl/data-sets. Accessed 01 Mar 2018

  58. Ojansivu, V., Heikkilä, J.: Blur insensitive texture classification using local phase quantization. In: International Conference on Image and Signal Processing, pp. 236–243 (2008)

    Google Scholar 

  59. Ojansivu, V., Rahtu, E., Heikkilä, J.: Rotation invariant local phase quantization for blur insensitive texture analysis. In: International Conference on Pattern Recognition, pp. 1–4. IEEE (2008)

    Google Scholar 

  60. Omara, I., Wu, X., Zhang, H., Du, Y., Zuo, W.: Learning pairwise SVM on hierarchical deep features for ear recognition. IET Biom. (2018)

    Google Scholar 

  61. Pflug, A., Busch, C., Ross, A.: 2D ear classification based on unsupervised clustering. In: International Joint Conference on Biometrics, pp. 1–8. IEEE (2014)

    Google Scholar 

  62. Pflug, A., Busch, C.: Ear biometrics: a survey of detection, feature extraction and recognition methods. IET Biom. 1(2), 114–129 (2012)

    Article  Google Scholar 

  63. Pflug, A., Paul, P.N., Busch, C.: A comparative study on texture and surface descriptors for ear biometrics. In: International Carnahan Conference on Security Technology, pp. 1–6. IEEE (2014)

    Google Scholar 

  64. Pflug, A., Winterstein, A., Busch, C.: Robust localization of ears by feature level fusion and context information. In: International Conference on Biometrics, pp. 1–8 (2013)

    Google Scholar 

  65. Phillips, P.J., Wechsler, H., Huang, J., Rauss, P.J.: The FERET database and evaluation procedure for face-recognition algorithms. Image Vis. Comput. 16(5), 295–306 (1998)

    Article  Google Scholar 

  66. Pietikäinen, M., Hadid, A., Zhao, G., Ahonen, T.: Computer Vision Using Local Binary Patterns. Computational Imaging and Vision. Springer (2011)

    Google Scholar 

  67. Prakash, S., Gupta, P.: An efficient ear localization technique. Image Vis. Comput. 30(1), 38–50 (2012)

    Article  Google Scholar 

  68. Prakash, S., Gupta, P.: An efficient ear recognition technique invariant to illumination and pose. Telecommun. Syst. 52(3), 1435–1448 (2013)

    Article  Google Scholar 

  69. Prakash, S., Gupta, P.: Ear Biometrics in 2D and 3D: Localization and Recognition, vol. 10. Springer (2015)

    Google Scholar 

  70. Prakash, S., Jayaraman, U., Gupta, P.: Ear localization from side face images using distance transform and template matching. In: Workshops on Image Processing Theory, Tools and Applications, pp. 1–8 (2008)

    Google Scholar 

  71. Prakash, S., Jayaraman, U., Gupta, P.: Connected component based technique for automatic ear detection. In: International Conference on Image Processing, pp. 2741–2744. IEEE (2009)

    Google Scholar 

  72. Prakash, S., Jayaraman, U., Gupta, P.: A skin-color and template based technique for automatic ear detection. In: International Conference on Advances in Pattern Recognition, pp. 213–216. IEEE (2009)

    Google Scholar 

  73. Proença, H., Alexandre, L.A.: The NICE.I: Noisy iris challenge evaluation—Part I. In: International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–4. IEEE (2007)

    Google Scholar 

  74. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  75. Sarangi, P.P., Panda, M., Mishra, B.S.P., Dehuri, S.: An automated ear localization technique based on modified Hausdorff distance. In: International Conference on Computer Vision and Image Processing, pp. 1–12 (2016)

    Google Scholar 

  76. Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: International Conference on Automatic Face and Gesture Recognition, pp. 53–58. IEEE (2002)

    Google Scholar 

  77. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)

  78. Szkuta, B.R., Sanabria, L.A., Dillon, T.S.: Electricity price short-term forecasting using artificial neural networks. IEEE Trans. Power Syst. 14(3), 851–857 (1999)

    Article  Google Scholar 

  79. Tian, L., Mu, Z.: Ear recognition based on deep convolutional network. In: International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 437–441. IEEE (2016)

    Google Scholar 

  80. University of Sheffield: The Sheffield (previously UMIST) face database (1998). https://www.sheffield.ac.uk/eee/research/iel/research/face. Accessed 01 May 2016

  81. Urooj, A., Borji, A.: Analysis of hand segmentation in the wild. In: Conference on Computer Vision and Pattern Recognition, IEEE. pp. 4710–4719 (2018)

    Google Scholar 

  82. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Conference on Computer Vision and Pattern Recognition, pp. I–I. IEEE (2001)

    Google Scholar 

  83. Vu, N.S., Caplier, A.: Face recognition with patterns of oriented edge magnitudes. In: European Conference on Computer Vision, pp. 313–326 (2010)

    Chapter  Google Scholar 

  84. Wahab, N.K.A., Hemayed, E.E., Fayek, M.B.: HEARD: an automatic human ear detection technique. In: International Conference on Engineering and Technology, pp. 1–7 (2012)

    Google Scholar 

  85. Zhang, Y., Mu, Z., Yuan, L., Yu, C.: Ear verification under uncontrolled conditions with convolutional neural networks. IET Biom. 7(3), 185–198 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported in parts by the ARRS (Slovenian Research Agency) Research Program P2-0250 (B) Metrology and Biometric Systems, the ARRS Research Program P2-0214 (A) Computer Vision. The authors thank NVIDIA for donating the Titan Xp GPU that was used in the experiments and our colleague Blaž Meden for his help with RefineNet’s Matlab scripts.

Author information

Authors and Affiliations

  1. Computer Vision Laboratory, Faculty of Computer and Information Science, University of Ljubljana, Večna pot 113, 1000, Ljubljana, Slovenia

    Žiga Emeršič & Peter Peer

  2. Laboratory of Artificial Perception, Systems and Cybernetics, Faculty of Electrical Engineering, University of Ljubljana, Tržaška cesta 25, 1000, Ljubljana, Slovenia

    Janez Križaj & Vitomir Štruc

Authors
  1. Žiga Emeršič
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janez Križaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vitomir Štruc
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Peer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Žiga Emeršič .

Editor information

Editors and Affiliations

  1. Department of Computer Science, South Valley University, Luxor, Egypt

    Mahmoud Hassaballah

  2. Department of Information Technology, Zagazig University, Zagazig, Egypt

    Khalid M. Hosny

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Emeršič, Ž., Križaj, J., Štruc, V., Peer, P. (2019). Deep Ear Recognition Pipeline. In: Hassaballah, M., Hosny, K. (eds) Recent Advances in Computer Vision. Studies in Computational Intelligence, vol 804. Springer, Cham. https://doi.org/10.1007/978-3-030-03000-1_14

Download citation

Publish with us

Policies and ethics

Search

Navigation