Skip to main content
SpringerLink
Log in

Deep learning approach for bubble segmentation from hysteroscopic images

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Gas embolism is a potentially serious complication of hysteroscopic surgery. It is particularly necessary to monitor bubble parameters in hysteroscopic images by computer vision method for helping develop automatic bubble removal devices. In this work, a framework combining a deep edge-aware network and marker-controlled watershed algorithm is presented to extract bubble parameters from hysteroscopy images. The proposed edge-aware network consists of an encoder-decoder architecture for bubble segmentation and a contour branch which is supervised by edge losses. The post-processing method based on marker-controlled watershed algorithm is used to further separate bubble instances and calculate size distribution. Extensive experiments substantiate that the proposed model achieves better performance than some typical segmentation methods. Accuracy, sensitivity, precision, Dice score, and mean intersection over union (mean IoU) obtained for the proposed edge-aware network are observed as 0.859 ± 0.017, 0.868 ± 0.019, 0.955 ± 0.005, 0.862 ± 0.005, and 0.758 ± 0.007, respectively. This work provides a valuable reference for automatic bubble removal devices in hysteroscopic surgery.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Umranikar S, Umranikar A, Cheong Y (2010) Hysteroscopy and hysteroscopic surgery. Obstet Gynaecol Reprod Med 20(6):167–173. https://doi.org/10.1016/j.ogrm.2010.03.006

    Article  Google Scholar 

  2. Groenman FA, Peters LW, Rademaker BMP, Bakkum EA (2008) Embolism of air and gas in hysteroscopic procedures: pathophysiology and implication for daily practice. J Minim Invasive Gynecol 15(2):241–247. https://doi.org/10.1016/j.jmig.2007.10.010

    Article  PubMed  Google Scholar 

  3. Sabsovich I, Abel M, Lee CJ, Spinelli AD, Abramowicz AE (2012) Air embolism during operative hysteroscopy: TEE-guided resuscitation. J Clin Anesth 24(6):480–486. https://doi.org/10.1016/j.jclinane.2012.01.002

    Article  PubMed  Google Scholar 

  4. Vilos GA, Hutson JR, Singh IS, Giannakopoulos F, Vilos AG (2019) Venous gas embolism during hysteroscopic endometrial ablation: report of five cases and review of the literature. J Minim Invasive Gynecol 27(3):748–754. https://doi.org/10.1016/j.jmig.2019.05.003

    Article  PubMed  Google Scholar 

  5. Corson SL, Brooks PG, Soderstrom RM (1996) Gynecologic endoscopic gas embolism. Fertil Steril 65(3):529–533. https://doi.org/10.1016/S0015-0282(16)58148-5

    Article  CAS  PubMed  Google Scholar 

  6. Dyrbye BA, Overdijk LE, van Kesteren PJ, de Haan P, Riezebos RK, Bakkum EA, Rademaker BM (2012) Gas embolism during hysteroscopic surgery using bipolar or monopolar diathermia: a randomized controlled trial. Am J Obstet Gynecol 207(4):271.e1-271.e6. https://doi.org/10.1016/j.ajog.2012.07.027

    Article  Google Scholar 

  7. Munro MG, Weisberg M, Rubinstein E (2001) Gas and air embolization during hysteroscopic electrosurgical vaporization: comparison of gas generation using bipolar and monopolar electrodes in an experimental model. J Am Assoc Gynecol Laparosc 8:488–494. https://doi.org/10.1016/S1074-3804(05)60609-3

    Article  CAS  PubMed  Google Scholar 

  8. Otsu N (2007) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66. https://doi.org/10.1109/TSMC.1979.4310076

    Article  Google Scholar 

  9. Vincent L, Soille P (1991) Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans Pattern Anal Mach Intell 13(6):583–598. https://doi.org/10.1109/34.87344

    Article  Google Scholar 

  10. Jahedsaravani A, Massinaei M, Marhaban MH (2017) An image segmentation algorithm for measurement of flotation froth bubble size distributions. Measurement 111:29–37. https://doi.org/10.1016/j.measurement.2017.07.023

    Article  Google Scholar 

  11. Zhang H, Tang Z, Xie Y, Gao X, Chen Q (2019) A watershed segmentation algorithm based on an optimal marker for bubble size measurement. Measurement 138:182–193. https://doi.org/10.1016/j.measurement.2019.02.005

    Article  Google Scholar 

  12. Ni J, Wu J, Tong J, Chen Z, Zhao J (2020) GC-Net: global context network for medical image segmentation. Comput Methods Programs Biomed 190:105121. https://doi.org/10.1016/j.cmpb.2019.105121

    Article  PubMed  Google Scholar 

  13. Noh H Hong S Han B (2015) Learning deconvolution network for semantic seg- mentation, in: Proceedings of the IEEE International Conference on Computer Vision, 1520–1528. https://doi.org/10.1109/ICCV.2015.178

  14. Ciresan D Giusti A Gambardella LM Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images, in: Advances in neural information processing systems, 2843–2851.

  15. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. IEEE T Pattern Anal 39(4):640–651. https://doi.org/10.1109/CVPR.2015.7298965

    Article  Google Scholar 

  16. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional en- coder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615

    Article  PubMed  Google Scholar 

  17. Zhao H Shi J Qi X Wang X Jia J (2017) Pyramid scene parsing network, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2881–2890. https://doi.org/10.1109/CVPR.2017.660

  18. Chen L Zhu Y Papandreou G Schroff F Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation, in Proceedings of the European Conference on Computer Vision (ECCV), 801–818. https://doi.org/10.1007/978-3-030-01234-2_49

  19. Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  PubMed  Google Scholar 

  20. Fu J Liu J Tian H Li Y Bao Y Fang Z Lu H (2019) Dual attention network for scene segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3146–3154. https://doi.org/10.1109/CVPR.2019.00326

  21. Ronneberger O Fischer P Brox T (2015) U-Net: convolutional networks for biomedical image segmentation, in Proceedings of the International Conference on Medical image Computing and Computer-Assisted Intervention, 234–241. https://arxiv.org/abs/1505.04597

  22. Milletari F Navab N Ahmadi SA (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. in Proceedings of the 2016 Fourth International Conference on3D Vision (3DV), 565–571. https://doi.org/10.1109/3DV.2016.79

  23. Çiçek Ö Abdulkadir A Lienkamp SS Brox T Ronneberger O (2016) 3d U-Net: learning dense volumetric segmentation from sparse annotation. in Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 424–432. https://doi.org/10.1007/978-3-319-46723-8_49

  24. He K Zhang X Ren S Sun J (2016) Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778. https://doi.org/10.1109/CVPR.2016.90

  25. Chidester B Ton TV Tran MT Ma J Do MN (2019) Enhanced rotation-equivariant u-net for nuclear segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. https://doi.org/10.1109/CVPRW.2019.00143

  26. Wang G Li W Ourselin S Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In Proceedings of the International MICCAI Brainlesion Workshop. https://doi.org/10.1007/978-3-319-75238-9_16

  27. Liu D Zhang H Zhao MM Yu XJ Yao SW Zhou W (2018) Brain tumor segmentation based on dilated convolution refine network. In Proceedings of the International Conference on Software Engineering Research, Management and Applications.18161668. https://doi.org/10.1109/SERA.2018.8477213

  28. Tseng KL Lin YL Hsu W Huang CY (2017) Joint sequence learning and cross-modality convolution for 3d biomedical segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2017.398

  29. Chen H Qi X Yu L Dou Q Qin J Heng PA (2017) Dcan: deep contour-aware networks for object instance segmentation from histology images. Med. Image Anal. 36 (Supplement C):135–146. https://doi.org/10.1016/j.media.2016.11.004

  30. Zhou Y Onder OF Dou Q Tsougenis E Chen H Heng PA (2019) Cia-net: robust nuclei instance segmentation with contour-aware information aggregation, in Proceedings of International Conference on Information Processing in Medical Imaging, 682–693.

  31. Murugesan B Sarveswaran K Shankaranarayana SM Ram K Joseph J Sivaprakasam M (2019) Psi-Net: shape and boundary aware joint multi-task deep network for medical image segmentation. In Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society, 7223-7226. https://doi.org/10.1109/EMBC.2019.8857339

  32. Zhang Z Fu H Dai H Shen J Pang Y Shao L (2019) ET-Net: a generic edge-attention guidance network for medical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 442–450.

  33. Hatamizadeh A Terzopoulos D Myronenko A (2019) End-to-end boundary aware networks for medical image segmentation, In Proceedings of the MICCAI Machine Learning in Medical Imaging.

  34. Pech-Pacheco JL Cristobal G Chamorro-Martinez J Fernandez-Valdivia J (2000) Diatom autofocusing in brightfield microscopy: a comparative study, in Proceedings of 15th International Conference on. IEEE Computer Society. https://doi.org/10.1109/ICPR.2000.903548

  35. Suzuki S, Be K (1985) Topological structural analysis of digitized binary images by border following. Comput Vision Graph Image Process 30(1):32–46. https://doi.org/10.1016/0734-189X(85)90016-7

    Article  Google Scholar 

  36. Soille PJ, Ansoult MM (1990) Automated basin delineation from digital elevation models using mathematical morphology. Signal Process 20(2):171–182. https://doi.org/10.1016/0165-1684(90)90127-K

    Article  Google Scholar 

  37. Beucher S Meyer F (1993) The morphological approach to segmentation: the watershed transform, mathematical morphology in image processing, 433–481. https://doi.org/10.1201/9781482277234-12

  38. Abadi M Barham P Chen J Chen Z Davis A Dean J Devin M Ghemawat S Irving G Isard M Kudlur M Levenberg J Monga R Moore S Murray DG Steiner B Tucker P Vasudevan V Warden P Wicke M Yu Y Zheng X (2016), Tensorflow: a system for large-scale machine learning. https://arxiv.org/abs/1605.08695

  39. Kermi A Mahmoudi I Khadir MT (2018) Deep convolutional neural networks using U-Net for automatic brain tumor segmentation in multimodal MRI volumes. In Proceedings of the International MICCAI Brainlesion.

Download references

Funding

This study was the financially supported by the National Natural Science Foundation of China (No. 52075325). The project is also supported by the “cross research fund for translational medicine” of Shanghai Jiaotong University (zh2018qnb17, zh2018qna37).

Author information

Authors and Affiliations

  1. State Key Laboratory of Mechanical Systems and Vibration, Shanghai Jiao Tong University, Shanghai, 200240, China

    Dong Wang, Wei Dai, Ding Tang, Huamiao Wang & Yinghong Peng

  2. School of Medicine, The International Peace Maternity and Child Health Hospital, Shanghai Jiao Tong University, Shanghai, China

    Yan Liang & Jing Ouyang

Authors
  1. Dong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ding Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jing Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huamiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yinghong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yan Liang or Jing Ouyang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Dai, W., Tang, D. et al. Deep learning approach for bubble segmentation from hysteroscopic images. Med Biol Eng Comput 60, 1613–1626 (2022). https://doi.org/10.1007/s11517-022-02562-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02562-8

Keywords

Search

Navigation