Skip to main content
Log in

Gaussian affinity and GIoU-based loss for perforation detection and localization from colonoscopy videos

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

A Correction to this article was published on 13 April 2023

This article has been updated

Abstract

Purpose

Endoscopic submucosal dissection (ESD) is a minimally invasive treatment for early gastric cancer. However, perforations may happen and cause peritonitis during ESD. Thus, there is a potential demand for a computer-aided diagnosis system to support physicians in ESD. This paper presents a method to detect and localize perforations from colonoscopy videos to avoid perforation ignoring or enlarging by ESD physicians.

Method

We proposed a training method for YOLOv3 by using GIoU and Gaussian affinity losses for perforation detection and localization in colonoscopic images. In this method, the object functional contains the generalized intersection over Union loss and Gaussian affinity loss. We propose a training method for the architecture of YOLOv3 with the presented loss functional to detect and localize perforations precisely.

Results

To qualitatively and quantitatively evaluate the presented method, we created a dataset from 49 ESD videos. The results of the presented method on our dataset revealed a state-of-the-art performance of perforation detection and localization, which achieved 0.881 accuracy, 0.869 AUC, and 0.879 mean average precision. Furthermore, the presented method is able to detect a newly appeared perforation in 0.1 s.

Conclusions

The experimental results demonstrated that YOLOv3 trained by the presented loss functional were very effective in perforation detection and localization. The presented method can quickly and precisely remind physicians of perforation happening in ESD. We believe a future CAD system can be constructed for clinical applications with the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Change history

References

  1. Ferlay J, Steliarova-Foucher E, Lortet-Tieulent J, Rosso S, Coebergh J-WW, Comber H, Forman D, Bray Freddie (2013) Cancer incidence and mortality patterns in Europe: estimates for 40 countries in 2012. Eur J Cancer 49(6):1374–1403

    Article  CAS  PubMed  Google Scholar 

  2. Gotoda T, Yamamoto H, Soetikno Roy M (2006) Endoscopic submucosal dissection of early gastric cancer. J Gastroenterol 41(10):929–942

    Article  PubMed  Google Scholar 

  3. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779–788

  4. Tsai M-H, Chen W-J, Lin J-Y, Lin G-S, Yan S-L (2020) Polyp classification based on deep neural network for colonoscopic images. In: Proceedings of the 2020 the 4th international conference on graphics and signal processing, pp. 61–64

  5. Urban G, Tripathi P, Alkayali T, Mittal M, Jalali F, Karnes W, Baldi Pierre (2018) Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 155(4):1069–1078

    Article  PubMed  Google Scholar 

  6. Misawa M, Shin K, Yuichi M, Kinichi H, Kazuo O, Takahisa M, Shoichi S, Toyoki K, Toshiyuki B, Fumio I, Hayato I, Masahiro O, Kensaku M (2021) Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video). Gastrointest Endosc 93(4):960–967

    Article  PubMed  Google Scholar 

  7. Itoh H, Oda M, Jiang K, Mori Y, Misawa M, Kudo S-E, Imai K, Ito S, Hotta K, Mori K (2021) Binary polyp-size classification based on deep-learned spatial information. Int J Comput Assist Radiol Surg 16(10):1817–1828

    Article  PubMed  Google Scholar 

  8. Nogueira-Rodríguez A, Domínguez-Carbajales R, Campos-Tato F, Herrero J, Puga M, Remedios D, Rivas L, Sánchez E, Iglesias Á, Cubiella J, Fdez-Riverola F, López-Fernández H, Reboiro-Jato M, Glez-Peña Daniel (2022) Real-time polyp detection model using convolutional neural networks. Neural Comput Appl 34(13):10375–10396

    Article  Google Scholar 

  9. Itoh H, Misawa M, Mori Y, Kudo S-E, Oda M, Mori Kensaku (2022) Positive-gradient-weighted object activation mapping: visual explanation of object detector towards precise colorectal-polyp localisation. Int J Comput Assist Radiol Surg 17(11):2051–2063

    Article  PubMed  Google Scholar 

  10. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767

  11. Jiang K, Itoh H, Oda M, Okumura T, Mori Y, Misawa M, Hayashi T, Kudo S-E, Mori K (2021) Dense-layer-based YOLO-v3 for detection and localization of colon perforations. In: Medical imaging 2021: computer-aided diagnosis, vol. 11597, p. 115971A. International Society for Optics and Photonics

  12. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) YOLOv4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  13. Choi J, Chun D, Kim H, Lee H-J (2019) Gaussian YOLOv3: an accurate and fast object detector using localization uncertainty for autonomous driving. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 502–511

  14. Liu Z, Wang Song (2019) Broken corn detection based on an adjusted YOLO with focal loss. IEEE Access 7:68281–68289

  15. Zhaohui Z, Ping W, Wei L, Jinze L, Rongguang Y, Dongwei R (2020) Distance-IOU loss: faster and better learning for bounding box regression. In: Proceedings of the AAAI conference on artificial intelligence 34:12993–13000

  16. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: a metric and a loss for bounding box regression. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 658–666

  17. Hayat M, Khan S, Zamir S W, Shen J, Shao L (2019) Gaussian affinity for max-margin class imbalanced learning. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 6469–6479

  18. Wang X, Shrivastava A, Gupta A (2017) A-fast-RCNN: hard positive generation via adversary for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2606–2615

  19. Hosang J, Benenson R, Schiele B (2017) Learning non-maximum suppression. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4507–4515

  20. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Köpf A, Yang E, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: an imperative style, high-performance deep learning library. In: HM Wallach, H Larochelle, A Beygelzimer, F Buc, EB Fox, and R Garnett, (eds), NeurIPS, pp. 8024–8035

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  22. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708

  23. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp. 2980–2988

  24. Allen David M (1971) Mean square error of prediction as a criterion for selecting variables. Technometrics 13(3):469–475

    Article  Google Scholar 

  25. Goodfellow I, Bengio Y, Courville A (2016) 6.2. 2.3 softmax units for multinoulli output distributions. Deep Learn, (1):180

  26. Henderson P, Ferrari V (2016) End-to-end training of object class detectors for mean average precision. In: Asian conference on computer vision, pp. 198–213. Springer

  27. Kalpić D, Hlupić N, Lovrić M (2011) Student’s t-tests. Springer, Berlin, pp 1559–1563

    Google Scholar 

Download references

Acknowledgements

Part of this study was funded by grants from JSPS MEXT KAKENHI (17H00867), and the JSPS Bilateral Joint Research Project.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Kai Jiang or Kensaku Mori.

Ethics declarations

Conflict of interest

Kudo SE, Misawa M received lecture fees from Olympus. Mori Y received consultant and lecture fees from Olympus. Mori K is supported by Cybernet Systems and Olympus (research grant) in this work and by NTT outside of the submitted work. The other authors have no conflicts of interest.

Ethical approval

This study was in accordance with the ethical committee (Showa University Northern Yokohama Hospital No. 19H049, and Nagoya University Ethics Review Committee No. hc21-06).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: K. Jiang and H. Itoh and Kensaku Mori’s affiliations have been corrected.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, K., Itoh, H., Oda, M. et al. Gaussian affinity and GIoU-based loss for perforation detection and localization from colonoscopy videos. Int J CARS 18, 795–805 (2023). https://doi.org/10.1007/s11548-022-02821-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02821-x

Keywords

Navigation