Skip to main content
Springer Nature Link
Log in

Tympanic membrane segmentation in otoscopic images based on fully convolutional network with active contour loss

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This paper presents a method to automatically segment tympanic membranes (TMs) from video-otoscopic images based on the deep learning approach. The paper introduces a hybrid loss function combining the Dice loss and active contour loss to the fully convolutional network. By this way, the proposed model takes into account the Dice similarity and the desired boundary contour information including the contour length as well as regions inside and outside the contour during learning. The proposed loss function is then applied to the fully convolutional network for tympanic membrane segmentation. We evaluate the proposed approach on TMs data set which includes 1139 otoscopic images from patients diagnosed with and without otitis media. Experimental results show that the proposed deep learning model achieves an average Dice similarity coefficient of 0.895, a mean Hausdorff distance of 19.189, and average perpendicular distance of 6.429, that outperforms other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig.2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Jabarin, B., Pitaro, J., Lazarovitch, T., Gavriel, H., Muallem-Kalmovich, L., Eviatar, E., Marom, T.: Decrease in pneumococcal otitis media cultures with concomitant increased antibiotic susceptibility in the pneumococcal conjugate vaccines era. Otol. Neurotol. 38(6), 853–859 (2017)

    Article  Google Scholar 

  2. Shie, C., Chang, H., Fan, F., Chen, C., Fang, T., Wang, P.: A hybrid feature-based segmentation and classification system for the computer aided self-diagnosis of otitis media. In: Proceedings of Conference IEEE Engineering in Medicine and Biology Society, pp. 4655–4658 (2014)

  3. Fang, T., Rafai, E., Wang, P., Bai, C., Jiang, P., Huang, S.N., Chen, Y., Chao, Y., Wang, C., Chang, C.: Pediatric otitis media in Fiji: survey findings 2015. Int. J. Ped. Otorhinolaryngol. 85, 50–55 (2016)

    Article  Google Scholar 

  4. Lieberthal, A., Carroll, A., Chonmaitree, T., Ganiats, T., Hoberman, A., Jackson, M., Joffe, M., Miller, D., Rosenfeld, R., Sevilla, X., Schwartz, R., Thomas, P., Tunkel, D.: The diagnosis and management of acute otitis media. Pediatrics 131(3), e964–e999 (2013)

    Article  Google Scholar 

  5. Jaisinghani, V., Hunter, L., Li, Y., Margolis, R.: Quantitative analysis of tympanic membrane disease using video-otoscopy. Laryngoscope 110(10 Pt 1), 1726–1730 (2000)

    Article  Google Scholar 

  6. Comunello, E., Wangenheim, A., Junior, V., Dornelles, C., Costa, S.: A computational method for the semi-automated quantitative analysis of tympanic membrane perforations and tympanosclerosis. Comput. Biol. Med. 39(10), 889–895 (2009)

    Article  Google Scholar 

  7. Tran, T., Fang, T., Pham, V., Lin, C., Wang, P., Lo, M.: Development of an automatic diagnostic algorithm for pediatric otitis media. Otol. Neurotol. 39(8), 1060–1065 (2018)

    Article  Google Scholar 

  8. Hsu, C., Chen, Y., Hwang, J., Liu, T.: A computer program to calculate the size of tympanic membrane perforations. Clin. Otolaryngol. Allied Sci. 29(4), 340–342 (2004)

    Article  Google Scholar 

  9. Ibekwe, T., Adeosun, A., Nwaorgu, O.: Quantitative analysis of tympanic membrane perforation: a simple and reliable method. J. Laryngol. Otol. 123(1), e2 (2009)

    Article  Google Scholar 

  10. Xie, X., Mirmehdi, M., Richard Maw, R., Amanda Hall, A.: Detecting abnormalities in tympanic membrane images. In: Proceedings of the 9th Medical Image Understanding and Analysis, pp. 19–22 (2005)

  11. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015)

  12. Huang, L., Zhao, Y.G., Yang, T.J.: Skin lesion segmentation using object scale-oriented fully convolutional neural networks. SIViP 13(3), 431–438 (2019)

    Article  Google Scholar 

  13. Öztürk, S., Özkaya, U., Akdemir, B., Seyfi, L.: Convolution kernel size effect on convolutional neural network in histopathological image processing applications. In: International Symposium on Fundamentals of Electrical Engineering (ISFEE), Bucharest (2018)

  14. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Imaging and Computer-Assisted Intervention, pp. 234–241 (2015)

  15. Tran, P.V.: A fully convolutional neural network for cardiac segmentation in short-axis MRI. https://arxiv.org/abs/1604.00494 (2016)

  16. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  17. Li, L., Zhao, X., Lu, W., Tan, S.: Deep learning for variational multimodality tumor segmentation in PET/CT. Neurocomputing (2019). https://doi.org/10.1016/j.neucom.2018.1010.1099

    Article  Google Scholar 

  18. Duan, J., Schlemper, J., Bai, W., Dawes, J.W., Bello, G.T., Doumou, G., De Marvao, A., O’Regan, D.P., Rueckert, D.: Deep nested level sets: fully automated segmentation of cardiac MR images in patients with pulmonary hypertension. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 595–603 (2018)

  19. Avendi, M.R., Kheradvar, A., Jafarkhani, H.: A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI. Med. Image Anal. 30, 108–119 (2016)

    Article  Google Scholar 

  20. Öztürk, S., Akdemir, B.: A convolutional neural network model for semantic segmentation of mitotic events in microscopy images. Neural Comput. Appl. 31(8), 3719–3728 (2019)

    Article  Google Scholar 

  21. Öztürk, Ş., Akdemir, B.: Cell-type based semantic segmentation of histopathological images using deep convolutional neural networks. Int. J. Imaging Syst. Technol. 29(3), 234–246 (2019)

    Article  Google Scholar 

  22. Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 4th International Conference on 3D Vision (3DV), pp. 565–571 (2016)

  23. Chen, X., Williams, B.M., Vallabhaneni, S.R., Czanner, G., Williams, R., Zheng, Y.: Learning active contour models for medical image segmentation. In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11623–11640 (2019)

  24. Boykov, Y., Lee, V.S., Rusinek, H., Bansal, R.: Segmentation of dynamic N–D data sets via graph cuts using Markov models. In: Proceedings of International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), pp. 1058–1066 (2001)

  25. Rezaee, M., van der Zwet, P., Lelieveldt, B., van der Geest, R., Reiber, J.: A multiresolution image segmentation technique based on pyramidal segmentation and fuzzy clustering. IEEE Trans. Image Process. 9(7), 1238–1248 (2000)

    Article  Google Scholar 

  26. Li, C., Xu, C., Gui, C., Fox, M.D.: Distance regularized level set evolution and its application to image segmentation. IEEE Trans. Image Process. 19(12), 3243–3254 (2010)

    Article  MathSciNet  Google Scholar 

  27. Tran, T.T., Pham, V.T., Shyu, K.K.: Zernike moment and local distribution fitting fuzzy energy-based active contours for image segmentation. SIViP 8(1), 11–25 (2014)

    Article  Google Scholar 

  28. Duan, J., Pan, Z., Yin, X., Wei, W., Wang, G.: Some fast projection methods based on Chan–Vese model for image segmentation. EURASIP J. Image Video Process. 7, 7 (2014)

    Article  Google Scholar 

  29. Chan, T., Vese, L.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    Article  Google Scholar 

  30. He, L., Peng, Z., Everding, B., Wang, X., Han, C., Weiss, K., Wee, W.G.: A comparative study of deformable contour methods on medical image segmentation. Image Vis. Comput. 26(2), 141–163 (2008)

    Article  Google Scholar 

  31. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 1(4), 321–331 (1988)

    Article  Google Scholar 

  32. Sethian, J.A.: Level Set Methods and Fast Marching Methods. Cambridge University Press, Cambridge (1999)

    MATH  Google Scholar 

  33. Bresson, X., Esedoḡlu, S., Vandergheynst, P., Thiran, J., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis. 28(2), 151–167 (2007)

    Article  MathSciNet  Google Scholar 

  34. Tohka, J.: Surface extraction from volumetric images using deformable meshes: a comparative study. In: Proceedings of the 7th European Conference in Computer Vision, pp. 350–364 (2002)

  35. Chan, T., Sandberg, Y., Vese, L.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (2000)

    Article  Google Scholar 

  36. Wang, P., Chang, Y., Chuang, L., Su, H., Li, C.: Incidence and recurrence of acute otitis media in Taiwan’s pediatric population. Clinics 66(3), 395–399 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

This research is funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number 102.05-2018.302.

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Hanoi University of Science and Technology, Hanoi, Vietnam

    Van-Truong Pham & Thi-Thao Tran

  2. Department of Biomedical Sciences and Engineering, National Central University, Taoyüan, Taiwan

    Van-Truong Pham, Thi-Thao Tran & Men-Tzung Lo

  3. Department of Otolaryngology, Cathay General Hospital, Taipei, Taiwan

    Pa-Chun Wang

  4. School of Medicine, Fu Jen Catholic University, Taipei, Taiwan

    Pa-Chun Wang

Authors
  1. Van-Truong Pham
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Thi-Thao Tran
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Pa-Chun Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Men-Tzung Lo
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Thi-Thao Tran or Men-Tzung Lo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pham, VT., Tran, TT., Wang, PC. et al. Tympanic membrane segmentation in otoscopic images based on fully convolutional network with active contour loss. SIViP 15, 519–527 (2021). https://doi.org/10.1007/s11760-020-01772-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-020-01772-7

Keywords