Skip to main content

Advertisement

SpringerLink
Log in

Mitotic cell detection in histopathological images of neuroendocrine tumors using improved YOLOv5 by transformer mechanism

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Automatic analysis of pathological images is important for the diagnosis and treatment of diseases. The use of computerized systems in this field is becoming increasingly common. Due to evolving technology and the speed of information needed, it is desirable for computers to be able to recognize objects like humans. Deep learning methods, which are a subfield of artificial intelligence, and image processing algorithms that recognize objects from images have been used in many fields in recent years, including healthcare. The aim of this study is to detect the mitoses in the histopathological images of neuroendocrine tumors using image processing methods based on deep learning. In our study, You Only Look Once-v5 (YOLOv5), the most widely used object recognition method, was used by combining the YOLOv5 transform module. YOLOv5 recognized mitotic cells with an accuracy of 0.80, a recall of 0.67, and an F1 score of 0.73, while the YOLOv5 transformer model recognized mitotic cells with an accuracy of 0.89, a recall of 0.68, and an F1 score of 0.77. The acceleration of the process and the objective evaluation will contribute significantly to an accurate and fast diagnosis. Another advantage is the time saved for pathologists, who can concentrate on important cases. In summary, automatic mitotic cell detection will facilitate tumor grade determination, treatment, and patient monitoring.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Availability of data and materials

The data supporting the conclusions of this article are included within the article. Any queries regarding these data may be directed to the corresponding author.

Notes

  1. https://github.com/tzutalin/labelImg.git/ Last visited: 10-10-2022.

  2. http://colab.research.google.com Last visited: 13-10-2022.

  3. https://pytorch.org/hub/ultralytics yolov5/ Last visited: 09-09-2022.

References

  1. Sampedro-Carrillo, E.A.: Sample preparation and fixation for histology and pathology. In: Immunohistochemistry and Immunocytochemistry, pp. 33–45 (2022)

  2. Bhargava, R., Madabhushi, A.: Emerging themes in image informatics and molecular analysis for digital pathology. Annu. Rev. Biomed. Eng. 18, 387 (2016)

    Article  Google Scholar 

  3. Wick, M.R.: The hematoxylin and eosin stain in anatomic pathology—an often-neglected focus of quality assurance in the laboratory. In: Seminars in diagnostic pathology, vol 36, pp 303–311. Elsevier (2019)

  4. Gridley, M.F.: Manual of Histologic and Special Staining Technics. Armed Forces Institute of Pathology, USA (1957).

  5. Hayakawa, T., Prasath, V., Kawanaka, H., Aronow, B.J., Tsuruoka, S.: Computational nuclei segmentation methods in digital pathology: a survey. Arch. Comput. Methods Eng. 28(1), 1–13 (2021)

    Article  Google Scholar 

  6. Shah, A.A., Frierson, H.F., Jr., Cathro, H.P.: Analysis of immunohistochemical stain usage in different pathology practice settings. Am. J. Clin. Pathol. 138(6), 831–836 (2012)

    Article  Google Scholar 

  7. Bosman, F.T., Carneiro, F., Hruban, R.H., Theise, N.D., et al.: WHO Classification of Tumours of the Digestive System, vol. Ed. 4. World Health Organization (2010)

  8. Taheri, S., Golrizkhatami, Z.: Magnification-specific and magnification-independent classification of breast cancer histopathological image using deep learning approaches. Signal Image Video Process. 17(2), 1–9 (2022)

  9. Zhao, Z.-Q., Zheng, P., Xu, S.-T., Wu, X.: Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 30(11), 3212–3232 (2019)

    Article  Google Scholar 

  10. Rezazadeh, I., P. Duygulu.: Multi-task learning for gland segmentation. Signal Image Video Process. 17(1), 1–9 (2022)

  11. Deepika, J., Rajan, C., Senthil, T.: Improved CAPSNET model with modified loss function for medical image classification. SIViP 16(8), 2269–2277 (2022)

    Article  Google Scholar 

  12. Keren Evangeline, I., Glory Precious, J., Pazhanivel, N., Angeline Kirubha, S.: Automatic detection and counting of lymphocytes from immunohistochemistry cancer images using deep learning. J. Med. Biol. Eng. 40(5), 735–747 (2020)

    Article  Google Scholar 

  13. Liu, Z., Yang, C., Huang, J., Liu, S., Zhuo, Y., Lu, X.: Deep learning framework based on integration of s-mask r-cnn and inception-v3 for ultrasound image-aided diagnosis of prostate cancer. Futur. Gener. Comput. Syst. 114, 358–367 (2021)

    Article  Google Scholar 

  14. Sindhwani, N., Verma, S., Bajaj, T., Anand, R.: Comparative analysis of intelligent driving and safety assistance systems using yolo and ssd model of deep learning. Int. J. Inform. Syst. Model. Des. (IJISMD) 12(1), 131–146 (2021)

    Article  Google Scholar 

  15. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards realtime object detection with region proposal networks. Adv. Neural İnf. Process. Syst. 28, 91–99 (2015)

  16. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37 (2016). Springer

  17. Li, C., Wang, X., Liu, W., Latecki, L.J.: Deepmitosis: Mitosis detection via deep detection, verification and segmentation networks. Med. Image Anal. 45, 121–133 (2018)

    Article  Google Scholar 

  18. Chen, H., Wang, X., Heng, P.A.: Automated mitosis detection with deep regression networks. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1204–1207 (2016). IEEE

  19. Nateghi, R., Danyali, H., Helfroush, M.S.: Maximized inter-class weighted mean for fast and accurate mitosis cells detection in breast cancer histopathology images. J. Med. Syst. 41(9), 1–15 (2017)

    Article  Google Scholar 

  20. Alom, M.Z., Aspiras, T., Taha, T.M., Bowen, T., Asari, V.K.: Mitosisnet: end-to-end mitotic cell detection by multi-task learning. IEEE Access 8, 68695–68710 (2020)

    Article  Google Scholar 

  21. Sebai, M., Wang, T., Al-Fadhli, S.A.: Partmitosis: a partially supervised deep learning framework for mitosis detection in breast cancer histopathology images. IEEE Access 8, 45133–45147 (2020)

    Article  Google Scholar 

  22. Sohail, A., Khan, A., Wahab, N., Zameer, A., Khan, S.: A multiphase deep cnn based mitosis detection framework for breast cancer histopathological images. Sci. Rep. 11(1), 1–18 (2021)

    Article  Google Scholar 

  23. Zhang, X., Cornish, T.C., Yang, L., Bennett, T.D., Ghosh, D., Xing, F.: Generative adversarial domain adaptation for nucleus quantification in images of tissue immunohistochemically stained for ki-67. JCO Clin. Cancer Inform. 4, 666–679 (2020)

    Article  Google Scholar 

  24. Lei, H., Liu, S., Elazab, A., Gong, X., Lei, B.: Attention-guided multi-branch convolutional neural network for mitosis detection from histopathological images. IEEE J. Biomed. Health Inform. 25(2), 358–370 (2020)

    Article  Google Scholar 

  25. Hwang, M., Wu, C., Jiang, W.-C., Hung, W.-C.: A sequential attention interface with a dense reward function for mitosis detection. Int. J. Mach. Learn. Cybernet. 13, 2663–2675 (2022)

    Article  Google Scholar 

  26. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  27. Salman, M.E., Çakar, G.Ç., Azimjonov, J., Kösem, M., Cedi̇moğlu, İ. H.: Automated prostate cancer grading and diagnosis system using deep learning-based yolo object detection algorithm. Expert Syst. Appl. 201, 117148 (2022)

    Article  Google Scholar 

  28. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Adv. Neural İnf. Process. Syst. 30, 5998–6008 (2017)

  29. Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)

    Article  Google Scholar 

  30. Chlap, P., Min, H., Vandenberg, N., Dowling, J., Holloway, L., Haworth, A.: A review of medical image data augmentation techniques for deep learning applications. J. Med. Imaging Radiat. Oncol. 65(5), 545–563 (2021)

    Article  Google Scholar 

  31. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: Yolov4: Optimal speed and accuracy of object detection. http://arxiv.org/abs/2004.10934 (2020)

  32. Aljabri, M., AlAmir, M., AlGhamdi, M., Abdel-Mottaleb, M., ColladoMesa, F.: Towards a better understanding of annotation tools for medical imaging: a survey. Multimed. Tools Appl. 81(18), 25877–25911 (2022)

    Article  Google Scholar 

  33. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv preprint http://arxiv.org/abs/2010.11929 (2020)

  34. Guo, Z., Wang, C., Yang, G., Huang, Z., Li, G.: Msft-yolo: improved yolov5 based on transformer for detecting defects of steel surface. Sensors 22(9), 3467 (2022)

    Article  Google Scholar 

  35. Fawcett, T.: An introduction to roc analysis. Pattern Recogn. Lett. 27(8), 861–874 (2006)

    Article  MathSciNet  Google Scholar 

  36. Padilla, R., Passos, W.L., Dias, T.L., Netto, S.L., Da Silva, E.A.: A comparative analysis of object detection metrics with a companion open-source toolkit. Electronics 10(3), 279 (2021)

    Article  Google Scholar 

  37. Mahmood, T., Arsalan, M., Owais, M., Lee, M.B., Park, K.R.: Artificial intelligence-based mitosis detection in breast cancer histopathology images using faster r-cnn and deep cnns. J. Clin. Med. 9(3), 749 (2020)

    Article  Google Scholar 

  38. Zorgani, A., Mohamed, M., Mehmood, I., Ugail, H.: Deep yolo-based detection of breast cancer mitotic-cells in histopathological images. In: International Conference on Medical Imaging and Computer-Aided Diagnosis, pp. 335–342. Springer (2021)

Download references

Acknowledgements

This paper was carried out as a result of Zehra Yücel’s PhD studies.

Funding

This study was not supported by a foundation.

Author information

Authors and Affiliations

  1. Department of Computer Technologies, Necmettin Erbakan University, Konya, Turkey

    Zehra Yücel

  2. Graduate School of Science and Engineering, Hacettepe University, Ankara, Turkey

    Zehra Yücel

  3. Faculty of Engineering, Department of Computer Engineering, Hacettepe University, Ankara, Turkey

    Fuat Akal

  4. Faculty of Medicine, Department of Pathology, Necmettin Erbakan University, Konya, Turkey

    Pembe Oltulu

Authors
  1. Zehra Yücel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fuat Akal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pembe Oltulu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZY, PO, and FA generated the hypothesis. Sections were taken by ZY, PO, and FA, ZY and PO performed the creation of the datasets. ZY and FA created the deep learning models and performed the experiments.

Corresponding author

Correspondence to Zehra Yücel.

Ethics declarations

Conflict of interest

The authors declare that he has no conflict of interest.

Ethical approval

This study was in accordance with the ethical standards of the institutional and/or national research committee (The Ethics Approval Certificate of Hacettepe University Non-Interventional Clinical Research Ethics Commission numbered 16969557–1579) and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yücel, Z., Akal, F. & Oltulu, P. Mitotic cell detection in histopathological images of neuroendocrine tumors using improved YOLOv5 by transformer mechanism. SIViP 17, 4107–4114 (2023). https://doi.org/10.1007/s11760-023-02642-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02642-8

Keywords

Search

Navigation