Abstract
One of the main reasons for a half-sided facial paralysis is a dysfunction of the facial nerve. Physicians have to assess such a unilateral facial palsy with the help of standardized grading scales to evaluate the treatment. However, such assessments are usually very subjective and they are prone to variance and inconsistency between physicians due to their varying experience. We propose an automatic non-biased method using deep features combined with a linear regression method for facial palsy grading index prediction. With an extension of the free software tool Auto-eFace we annotated images of facial palsy patients and healthy subjects according to a common facial palsy grading scale. In our experiments, we obtained an average grading error of 11%.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Banks, C.A., Bhama, P.K., Park, J., Hadlock, C.R., Hadlock, T.A.: Clinician-graded electronic facial paralysis assessment: the eface. Plast. Reconstr. Surg. 136(2), 223e–230e (2015)
Barbosa, J., et al.: Efficient quantitative assessment of facial paralysis using iris segmentation and active contour-based key points detection with hybrid classifier. BMC Med. Imaging 16(1), 23 (2016)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)
Daugman, J.: How iris recognition works. In: The Essential Guide to Image Processing, pp. 715–739. Elsevier (2009)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Donahue, J., et al.: Decaf: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655 (2014)
Finsterer, J.: Management of peripheral facial nerve palsy. Eur. Arch. Otorhinolaryngol. 265(7), 743–752 (2008)
Gaber, A., Faher, M.F., Wahed, M.A.: Automated grading of facial paralysis using the kinect v2: a proof of concept study. In: International Conference on Virtual Rehabilitation, pp. 258–264. IEEE (2015)
Gaber, A., Taher, M.F., Wahed, M.A.: Quantifying facial paralysis using the kinect v2. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2497–2501. IEEE (2015)
Guarin, D.L., Dusseldorp, J., Hadlock, T.A., Jowett, N.: A machine learning approach for automated facial measurements in facial palsy. JAMA Facial Plast. Surg. 20(4), 335–337 (2018)
Guarin, D.L., et al.: Toward an automatic system for computer-aided assessment in facial palsy. arXiv preprint arXiv:1910.11497 (2019)
Haase, D., Kemmler, M., Guntinas-Lichius, O., Denzler, J.: Efficient measuring of facial action unit activation intensities using active appearance models. In: Machine Vision Applications, pp. 141–144 (2013)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE (2016)
House, J.W., Brackmann, D.E.: Facial nerve grading system. Otolaryngol. Head Neck Surg. 93(2), 146–147 (1985)
Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)
King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105 (2012)
Lankton, S., Tannenbaum, A.: Localizing region-based active contours. IEEE Trans. Image Process. 17(11), 2029–2039 (2008)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1449–1457 (2015)
Modersohn, L., Denzler, J.: Facial paresis index prediction by exploiting active appearance models for compact discriminative features. In: International Conference on Computer Vision Theory and Applications, pp. 271–278 (2016)
Morales, D.R., Donnan, P.T., Daly, F., Staa, T.V., Sullivan, F.M.: Impact of clinical trial findings on bell’s palsy management in general practice in the uk 2001–2012: interrupted time series regression analysis. BMJ Open 3(7), e003121 (2013)
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference, p. 6 (2015)
Plumbaum, K., et al.: Inpatient treatment of patients with acute idiopathic peripheral facial palsy: a population-based healthcare research study. Clin. Otolaryngol. 42(6), 1267–1274 (2017)
Ross, B.G., Fradet, G., Nedzelski, J.M.: Development of a sensitive clinical facial grading system. Otolaryngol. Head Neck Surg. 114(3), 380–386 (1996)
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: IEEE International Conference on Computer Vision - Workshops, pp. 397–403 (2013)
Schaede, R.A., Volk, G.F., Modersohn, L., Barth, J.M., Denzler, J., Guntinas-Lichius, O.: Video instruction for synchronous video recording of mimic movement of patients with facial palsy. Laryngo-Rhino-Otologie (2017)
Simon, M., Rodner, E.: Neural activation constellations: unsupervised part model discovery with convolutional networks. In: IEEE International Conference on Computer Vision, pp. 1143–1151. IEEE (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Stennert, E., Limberg, C., Frentrup, K.: An index for paresis and defective healing-an easily applied method for objectively determining therapeutic results in facial paresis (author’s transl). HNO 25(7), 238–245 (1977)
Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research. PMLR, Long Beach, California, USA, 09–15 June 2019, vol. 97, pp. 6105–6114 (2019)
Thielker, J., Geißler, K., Granitzka, T., Klingner, C., Volk, G., Guntinas-Lichius, O.: Acute management of bell’s palsy. Curr. Otorhinolaryngol. Rep. 6(2), 161–170 (2018)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995). https://doi.org/10.1007/978-1-4757-3264-1
Volk, G.F., et al.: Functional outcome and quality of life after hypoglossal-facial jump nerve suture. Front. Surg. 7, 11 (2020)
Volk, G.F., et al.: Reliability of grading of facial palsy using a video tutorial with synchronous video recording. The Laryngoscope 129(10), 2274–2279 (2019)
Acknowledgments
The research was supported by grant DE 735/15-1 and GU 463/12-1 of the German Research Foundation (DFG). We also gratefully acknowledge the support of NVIDIA Corporation with the donation of Titan Xp GPUs used for this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Raj, A., Mothes, O., Sickert, S., Volk, G.F., Guntinas-Lichius, O., Denzler, J. (2020). Automatic and Objective Facial Palsy Grading Index Prediction Using Deep Feature Regression. In: Papież, B., Namburete, A., Yaqub, M., Noble, J. (eds) Medical Image Understanding and Analysis. MIUA 2020. Communications in Computer and Information Science, vol 1248. Springer, Cham. https://doi.org/10.1007/978-3-030-52791-4_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-52791-4_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-52790-7
Online ISBN: 978-3-030-52791-4
eBook Packages: Computer ScienceComputer Science (R0)