Skip to main content

Advertisement

SpringerLink
Log in

Segmentation, Detection, and Classification of Cell Nuclei on Oral Cytology Samples Stained with Papanicolaou

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Although oral cancer is considered a global health issue with 350,000 people diagnosed over a year, it can successfully be treated if diagnosed at early stages. Papanicolaou is an inexpensive and non-invasive method, generally applied to detect cervical cancer, but it can also be useful to detect cancer on oral cavities. The manual process of analyzing cells to detect abnormalities is a time-consuming cell analysis and is subject to variations in perceptions from different professionals. This paper compares three different deep learning (DL) approaches: segmentation, object detection, and image classification. Our results show that the binary object detection with Faster R-CNN is the best approach for nuclei detection and localization (0.76 IoU). Since ResNet 34 had a good performance on abnormal nuclei classification (0.86 \(F_1\) score), we concluded that these two models can be used in combination to perform a reliable localization and classification pipeline. This work reinforces that the automated analysis of oral cytology to build a pipeline for nuclei classification and localization using DL can contribute to minimize the subjectivity of the human analysis and also support the detection of cancer at early stages.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. The dataset is available at https://arquivos.ufsc.br/d/5035aec3c24f421a95d0/

  2. This research was approved by the UFSC Research Ethics Committee (CEPSH), protocol number 23193719.5.0000.0121. All patients were previously approached and informed about the study objectives. Those who agreed to participate signed an Informed Consent Form.

References

  1. Amorim JG, Cerentini A, Macarini LAB, Matias AV, Wangenheim AV. Systematic literature review of computer vision-aided cytology - a review of classic computer vision and deep learning-based approaches published between january/2016 - march/2020. Tech. rep., Instituto Nacional para Convergência Digital - INCoD, 2020. https://doi.org/10.13140/RG.2.2.13304.67840. http://rgdoi.net/10.13140/RG.2.2.13304.67840.

  2. Andreóli Petrolini V, Beckhauser E, Savaris A, Ines Meurer M, von Wangenheim A, Krechel D. Collaborative telepathology in a statewide telemedicine environment—first tests in the context of the Brazilian public healthcare system. In: 2019 IEEE 32nd international symposium on computer-based medical systems (CBMS), 2019. pp 684–689. https://doi.org/10.1109/CBMS.2019.00139.

  3. Araújo FH, Silva RR, Ushizima DM, Rezende MT, Carneiro CM, Campos Bianchi AG, Medeiros FN. Deep learning for cell image segmentation and ranking. Comput Med Imaging Graphics. 2019;72:13–21. https://doi.org/10.1016/j.compmedimag.2019.01.003.

    Article  Google Scholar 

  4. Bell AA, Kaftan JN, Aach T, Meyer-Ebrecht D, Bocking A. High dynamic range images as a basis for detection of argyrophilic nucleolar organizer regions under varying stain intensities. In: 2006 international conference on image processing, IEEE. 2006. https://doi.org/10.1109/icip.2006.312959.

  5. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. Cancer J Clin. 2018;68(6):394–424. https://doi.org/10.3322/caac.21492.

    Article  Google Scholar 

  6. Carvalho L, Fauth G, Baecker Fauth S, Krahl G, Moreira A, Fernandes C, von Wangenheim A. Automated microfossil identification and segmentation using a deep learning approach. Mar Micropaleontol. 2020;158:101890. https://doi.org/10.1016/j.marmicro.2020.101890.

    Article  Google Scholar 

  7. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: CVPR09 2009.

  8. Deng J, Lu Y, Ke J. An accurate neural network for cytologic whole-slide image analysis. In: Proceedings of the Australasian computer science week multiconference, association for computing machinery, New York, NY, USA, ACSW ’20. 2020. https://doi.org/10.1145/3373017.3373039.

  9. Dey S, Sarkar R, Chatterjee K, Datta P, Barui A, Maity SP. Pre-cancer risk assessment in habitual smokers from DIC images of oral exfoliative cells using active contour and SVM analysis. Tissue Cell. 2017;49(2):296–306. https://doi.org/10.1016/j.tice.2017.01.009.

    Article  Google Scholar 

  10. Du, Li X, Li Q. Detection and classification of cervical exfoliated cells based on faster r-cnn*. In: 2019 IEEE 11th international conference on advanced infocomm technology (ICAIT), 2019. pp. 52–57. https://doi.org/10.1109/ICAIT.2019.8935931.

  11. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. CoRR abs/1512.03385. 2015. http://arxiv.org/abs/1512.03385.

  12. Howard J. 2019 deep learning 2019—Fastai Course. YouTube, Jan. 25 [Video file]. https://youtu.be/XfoYk_Z5AkI. Accessed 14 May 2020.

  13. Howard J, Gugger S. 2019 Fastai python library v1.0.57. http://docs.fast.ai/ 3. Accessed 3 Mar 2020.

  14. Kingma DP, Ba J. 2014 Adam: a method for stochastic optimization. Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015. http://arxiv.org/abs/1412.6980. arxiv:1412.6980Comment.

  15. Kolles H, Wangenheim AV. The use of neural network technology in automated grading of astrocytoma. Pathol Res Pract. 1997;194(4):254.

    Google Scholar 

  16. Kolles H, Wangenheim AV, Vince GH, Feiden W. Automatic grading of gliomas in stereotactic biopsies. Comparison of the classification results of neuronal networks and discriminant analysis. Clin Neuropathol. 1993;12(5):253.

    Google Scholar 

  17. Kolles H, Wangenheim AV, Niedermayer I, Vince GH, Feiden W. Computer assisted grading of gliomas of the astrocytoma/glioblastoma groups. Verh Dtsch Ges Pathol. 1994;78:427–31.

    Google Scholar 

  18. Kolles H, Wangenheim AV, Niedermayer I, Vince GH, Feiden W. Automated grading of astrocytomas based on histomorphometric analysis of ki-67 and feulgen stained paraffin sections. classification results of neuronal networks and discriminant analysis. Anal Cell Pathol. 1995;8(2):101–16.

    Google Scholar 

  19. Kolles H, Wangenheim AV, Rahmel J, Niedermayer I, Feiden W. Data-driven approaches to decision making in automated tumor grading. An example of astrocytoma grading. Anal Quant Cytol Histol. 1996;18(4):298–304.

    Google Scholar 

  20. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44. https://doi.org/10.1038/nature14539.

    Article  Google Scholar 

  21. Lin T, Maire M, Belongie SJ, Bourdev LD, Girshick RB, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL. 2014 Microsoft COCO: common objects in context. CoRR abs/1405.0312. http://arxiv.org/abs/1405.0312.

  22. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). 2017. https://doi.org/10.1109/cvpr.2017.106.

  23. Lin TY, Goyal P, Girshick R, He K, Dollar P. Focal loss for dense object detection. In: 2017 IEEE international conference on computer vision (ICCV) 2017. https://doi.org/10.1109/iccv.2017.324.

  24. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JA, van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88. https://doi.org/10.1016/j.media.2017.07.005.

    Article  Google Scholar 

  25. Lucena E, Miranda A, Araújo F, Galvão C, Medeiros A. Collection method and the quality of the smears from oral mucosa. Revista de Cirurgia e Traumatologia Buco-maxilo-facial. 2011;11(2):55–62.

    Google Scholar 

  26. Mehrotra R, Mishra S, Singh M, Singh M. The efficacy of oral brush biopsy with computer-assisted analysis in identifying precancerous and cancerous lesions. Head Neck Oncol. 2011;. https://doi.org/10.1186/1758-3284-3-39.

    Article  Google Scholar 

  27. Meurer MI, Von Wangenheim A, Zimmermann C, Savaris A, Petrolini VA, Wagner HM. Launching a public statewide tele(oral)medicine service in brazil during covid-19 pandemic. Oral Dis. 2020;. https://doi.org/10.1111/odi.13528.

    Article  Google Scholar 

  28. Nobre LF, von Wangenheim A, Ho K, Jarvis-Selinger S, Novak Lauscher H, Cordeiro J, Scott R. Development and implementation of a statewide telemedicine/telehealth system in the state of Santa Catarina, Brazil, Springer, New York; 2012. pp. 379–400. https://doi.org/10.1007/978-1-4614-3495-5_22.

  29. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B, Glocker B, Rueckert D. Attention u-net: learning where to look for the pancreas. 2018.

  30. Özgür Pektaş Z, Keskin A, Günhan Ömer, Karslioğlu Y. Evaluation of nuclear morphometry and DNA ploidy status for detection of malignant and premalignant oral lesions: Quantitative cytologic assessment and review of methods for cytomorphometric measurements. J Oral Maxillofac Surg. 2006;64(4):628–35. https://doi.org/10.1016/j.joms.2005.12.010.

    Article  Google Scholar 

  31. Ren S, He K, Girshick R, Sun J. Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2017;39(6):1137–49. https://doi.org/10.1109/tpami.2016.2577031.

    Article  Google Scholar 

  32. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. CoRR abs/1505.04597. 2015. http://arxiv.org/abs/1505.04597.

  33. Sergio BZ, Macarini LAB, Toé FPD, Wangenheim AV. Computer-assisted technologies for diagnosis of oral cancer on cytology samples—a systematic literature review. Technical report, Instituto Nacional para Convergência Digital—INCoD. 2019. https://doi.org/10.13140/RG.2.2.14207.76964.

  34. Smith LN. No more pesky learning rate guessing games. CoRR abs/1506.01186. 2015. http://arxiv.org/abs/1506.01186.

  35. Smith LN. A disciplined approach to neural network hyper-parameters: part 1—learning rate, batch size, momentum, and weight decay. CoRR abs/1803.09820. 2018. http://arxiv.org/abs/1803.09820.

  36. Solar M, Peña Gonzalez JP. Computational detection of cervical uterine cancer. In: 2019 sixth international conference on eDemocracy eGovernment (ICEDEG), 2019. pp. 213–217. https://doi.org/10.1109/ICEDEG.2019.8734400.

  37. Victória Matias A, Cerentini A, Buschetto Macarini LA, Atkinson Amorim JG, Perozzo Daltoé F, von Wangenheim A. Segmentation, detection and classification of cell nuclei on oral cytology samples stained with papanicolaou. In: 2020 IEEE 33rd international symposium on computer-based medical systems (CBMS), 2020. pp. 53–58. https://doi.org/10.1109/CBMS49503.2020.00018.

  38. Wang S, Yang DM, Rong R, Zhan X, Xiao G. Pathology image analysis using segmentation deep learning algorithms. Am J Pathol. 2019;189(9):1686–98. https://doi.org/10.1016/j.ajpath.2019.05.007.

    Article  Google Scholar 

  39. von Wangenheim A, Nunes DH. Creating a web infrastructure for the support of clinical protocols and clinical management: an example in teledermatology. Telemed e-Health. 2019;25(9):781–90. https://doi.org/10.1089/tmj.2018.0197 (pMID: 30499753).

    Article  Google Scholar 

  40. Wu Y, Kirillov A, Massa F, Lo WY, Girshick R. 2019. Detectron2. https://github.com/facebookresearch/detectron2. Accessed 3 Mar 2020.

  41. Zhang C, Liu D, Wang L, Li Y, Chen X, Luo R, Che S, Liang H, Li Y, Liu S, Tu D, Qi G, Luo P, Luo J. DCCL: a benchmark for cervical cytology analysis. In: Machine learning in medical imaging. Springer International Publishing, 2019. pp. 63–72. https://doi.org/10.1007/978-3-030-32692-0_8.

Download references

Acknowledgements

We would like to thank Dr. Felipe Perozzo Daltoé for providing the samples and Ricardo Thisted for labeling the images.

Author information

Authors and Affiliations

  1. Department of Informatics and Statistics, Federal University of Santa Catarina, Florianópolis, Brazil

    André Victória Matias, Allan Cerentini & João Gustavo Atkinson Amorim

  2. Automation and Systems Department, Federal University of Santa Catarina, Florianópolis, Brazil

    Luiz Antonio Buschetto Macarini

  3. Department of Pathology, Federal University of Santa Catarina, Florianópolis, Brazil

    Felipe Perozzo Daltoé

  4. Brazilian Institute for Digital Convergence, Federal University of Santa Catarina, Florianópolis, Brazil

    Aldo von Wangenheim

Authors
  1. André Victória Matias
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allan Cerentini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luiz Antonio Buschetto Macarini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. João Gustavo Atkinson Amorim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Felipe Perozzo Daltoé
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Aldo von Wangenheim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to André Victória Matias.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

We would like to thank Fundação de Amparo à Pesquisa e Inovação do Estado de Santa Catarina (Fapesc) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) for funding this work.

This article is part of the topical collection “AI and Deep Learning Trends in Healthcare” guest edited by KC Santosh, Paolo Soda and Zalelam Temesgen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Matias, A.V., Cerentini, A., Macarini, L.A.B. et al. Segmentation, Detection, and Classification of Cell Nuclei on Oral Cytology Samples Stained with Papanicolaou. SN COMPUT. SCI. 2, 285 (2021). https://doi.org/10.1007/s42979-021-00676-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-021-00676-8

Keywords

Search

Navigation