Skip to main content
SpringerLink
Log in

Face Symmetry Analysis Using a Unified Multi-task CNN for Medical Applications

  • Conference paper
  • First Online:
Intelligent Systems and Applications (IntelliSys 2018)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 869))

Included in the following conference series:

Abstract

Facial symmetry analysis can provide an important role in the diagnosis and rehabilitation of medical conditions like facial paralysis issues such as bell’s palsy. Recent advances in computer vision techniques specifically the use of deep convolutional neural networks and multi-task learning provide a gateway to fast and state-of-the-art accurate methods for object detection tasks. In this paper, we present a novel unified multi-task CNN framework for simultaneous object proposal, face detection and face symmetry analysis. We highlight the potential possibilities for the use of such a framework within the medical domain through the experimental results on two test data sets. The results are promising showing high level of accuracy for both the task of face detection and symmetry analysis while also highlighting the efficient computational overhead for our proposed method which can process an image in 0.04 s.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Grammer,K., Thornhill, R.: Human (Homo sapiens) facial attractiveness and sexual selection: the role of symmetry and averageness. J. Comp. Psychol. (Washington, D.C. : 1983) 108(3), 233–242 (1994)

    Article  Google Scholar 

  2. Ishii, L.E.: Facial Nerve Rehabilitation. Facial Plast. Surg. Clin. North Ame. 24(4), 573–575 (2016). [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/27712822 and http://linkinghub.elsevier.com/retrieve/pii/S1064740616300657

  3. Guerreschi, P., Gabert, P.-E., Labbé, D., Martinot-Duquennoy, V.: Paralysie faciale chez lenfant. Annales de Chirurgie Plastique Esthétique 61(5), 513–518 (2016). [Online]. Available http://www.ncbi.nlm.nih.gov/pubmed/27637411 and http://linkinghub.elsevier.com/retrieve/pii/S0294126016301340

    Article  Google Scholar 

  4. Monini, S., Buffoni, A., Romeo, M., Di Traglia, M., Filippi, C., Atturo, F., Barbara, M.: Kabat rehabilitation for Bells palsy in the elderly. Acta Oto-Laryngologica, 1–5 (2016). [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/27967288 and https://www.tandfonline.com/doi/full/10.1080/00016489.2016.1262553

  5. Lindsay, R.W., Robinson, M., Hadlock, T.A.: Comprehensive facial rehabilitation improves function in people with facial paralysis: a 5-year experience at the Massachusetts eye and ear infirmary. Phys. Ther 90(3), 391–397 (2010). [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/20093325 and https://academic.oup.com/ptj/article-lookup/doi/10.2522/ptj.20090176

  6. Banks, C.A., Bhama, P.K., Park, J., Hadlock, C.R., Hadlock, T.A.: Clinician-graded electronic facial paralysis assessment. Plast. Reconstr. Surg. 136(2), 223e–230e (2015). [Online]. Available: http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00006534-201508000-00029

  7. Wang, T., Dong, J., Sun, X., Zhang, S., Wang, S.: Automatic recognition of facial movement for paralyzed face. Biomed. Mater. Eng. 24(6), 2751–2760 (2014)

    Google Scholar 

  8. Wang, T., Zhang, S., Dong, J., Liu, L., Yu, H.: Automatic evaluation of the degree of facial nerve paralysis. Multimedia Tools Appl. 75(19), 11893–11908 (2016)

    Article  Google Scholar 

  9. Storey, G., Jiang, R., Bouridane, A.: Role for 2D image generated 3D face models in the rehabilitation of facial palsy. Healthc. Technol. Lett. 4(4), 145–148 (2017). [Online]. Available: http://digital-library.theiet.org/content/journals/10.1049/htl.2017.0023

  10. Yang, S., Luo, P., Loy, C.C., Tang, X.: From facial parts responses to face detection: a deep learning approach. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 11–18-Dece, no. 3, pp. 3676–3684 (2016)

    Google Scholar 

  11. Ranjan, R., Patel, V.M., Chellappa, R.: A deep pyramid deformable part model for face detection. In: 2015 IEEE 7th International Conference on Biometrics Theory, p. 2015. BTAS, Applications and Systems (2015)

    Google Scholar 

  12. Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild, pp. 2879–2886 (2012). [Online]. Available: http://dl.acm.org/citation.cfm?id=2354409.2355119

  13. Ranjan, R., Patel, V.M., Chellappa, R.: HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition (2016). [Online]. Available: http://arxiv.org/abs/1603.01249

  14. Yang, S., Luo, P., Loy, C.C., Tang, X.: WIDER FACE: a face detection benchmark. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-Dec, pp. 5525–5533 (2016)

    Google Scholar 

  15. Köstinger, M., Wohlhart, P., Roth, P.M., Bischof, H.: Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2144–2151 (2011)

    Google Scholar 

  16. Uijlings, J.R., Van De Sande, K.E., Gevers, T., Smeulders, A.W.: Selective search for object recognition. Int. J. Comput. Vis. 104(2), 154–171 (2013)

    Article  Google Scholar 

  17. Carreira, J., Sminchisescu, C.: CPMC: automatic object segmentation using constrained parametric min-cuts. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1312–1328 (2012)

    Article  Google Scholar 

  18. Zitnick, C.L., Doll, P.: Edge boxes: locating object proposals from edges. In: European Conference on Computer Vision, pp. 1–15 (2014)

    Google Scholar 

  19. Erhan, D., Szegedy, C., Toshev, A., Anguelov, D.: Scalable object detection using deep neural networks. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2155–2162 (2014). [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6909673

  20. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015). [Online]. Available: http://arxiv.org/abs/1506.01497

  21. Jones, M., Viola, P.: Fast multi-view face detection. Mitsubishi Electric Research Lab TR2000396, July 2003. [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.145.7598&rep=rep1&type=pdf

  22. Jonathon Phillips, P., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)

    Article  Google Scholar 

  23. Heisele, B., Serre, T., Poggio, T.: A component-based framework for face detection and identification. In. J. Comput. Vis. 74(2), 167–181 (2007)

    Article  Google Scholar 

  24. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004). [Online]. Available: http://link.springer.com/10.1023/B:VISI.0000013087.49260.fb

  25. Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2009). [Online]. Available: http://cs.brown.edu/~pff/papers/lsvm-pami.pdf and http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5255236

  26. Farfade, S.S., Saberian, M., Li, L.J.: Multi-view face detection using deep convolutional neural networks. In: International Conference on Multimedia Retrieval 2015 (ICMR), p. 19 (2015). [Online]. Available: http://arxiv.org/abs/1502.02766

  27. Yang, M.H., Kriegman, D.J., Ahuja, N.: Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(1), 34–58 (2002)

    Article  Google Scholar 

  28. Zhang, C., Zhang, Z.: A survey of recent advances in face detection. Microsoft Research, p. 17 (2010). [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.5270&rep=rep1&type=pdf

  29. Zafeiriou, S., Zhang, C., Zhang, Z.: A survey on face detection in the wild: past, present and future. Comput. Vis. Image Underst. 138, 1–24 (2015)

    Article  Google Scholar 

  30. Zhang, Z., Luo, P., Chen, C.L., Tang, X.: Facial landmark detection by deep multi-task learning. In: European Conference on Computer Vision, pp. 94–108 (2014)

    Google Scholar 

  31. Gkioxari, G., Hariharan, B., Girshick, R., Malik, J.: R-CNNs for Pose Estimation and Action Detection. arXiv preprint arXiv:1406.5212 (2014). [Online]. Available: http://arxiv.org/abs/1406.5212

  32. Wang,K., Luo, J.: Detecting visually observable disease symptoms from faces. EURASIP J. Bioinf. Syst. Biol. 2016(1), 13 (2016). [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/27688744 http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC5007273

  33. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The Extended Cohn-Kanade Dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 94–101. IEEE (2010). [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5543262

Download references

Acknowledgment

The authors would like to thank the financial support from the EPSRC grant (EP/P009727/1).

Author information

Authors and Affiliations

  1. Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, UK

    Gary Storey & Richard Jiang

Authors
  1. Gary Storey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Gary Storey or Richard Jiang .

Editor information

Editors and Affiliations

  1. Faculty of Science and Engineering, Saga University, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, UK

    Supriya Kapoor

  3. The Science and Information (SAI) Organization, Bradford, UK

    Rahul Bhatia

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Storey, G., Jiang, R. (2019). Face Symmetry Analysis Using a Unified Multi-task CNN for Medical Applications. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Intelligent Systems and Applications. IntelliSys 2018. Advances in Intelligent Systems and Computing, vol 869. Springer, Cham. https://doi.org/10.1007/978-3-030-01057-7_36

Download citation

Publish with us

Policies and ethics

Search

Navigation