Abstract
Interactive segmentation is of great importance in clinical practice for correcting and refining the automated segmentation by involving additional user hints, e.g., scribbles and clicks. Currently, interactive segmentation methods for 2D medical images are well studied, while seldom works are conducted on 3D medical volumetric data. Given a 3D volumetric image, the user interaction can only be performed on a few slices, thus the key issue is how to propagate the information over the entire volume for spatial-consistent segmentation. In this paper, we propose a novel hybrid propagation network for interactive segmentation of 3D medical images. Our proposed method consists of two key designs, including a slice propagation network (denoted as SPN) for transferring user hints to adjacent slices to guide the segmentation slice-by-slice and a volume propagation network (denoted as VPN) for propagating user hints over the entire volume in a global manner. Specifically, as for SPN, we adopt a memory-augmented network, which utilizes the information of segmented slices (memory slices) to propagate interaction information. To use interaction information propagated by VPN, a feature-enhanced memory module is designed, in which the volume segmentation information from the latent space of VPN is introduced into the memory module of SPN. In such a way, the interactive segmentation can leverage both advantages of volume and slice propagation, thus improving the volume segmentation results. We perform experiments on two commonly-used 3D medical datasets, with the experimental results indicating that our method outperforms the state-of-the-art methods. Our code is available at https://github.com/luyueshi/Hybrid-Propagation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 801–818 (2018)
Cheng, H.K., Tai, Y.W., Tang, C.K.: Modular interactive video object segmentation: interaction-to-mask, propagation and difference-aware fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5559–5568 (2021)
Cheng, H.K., Tai, Y.W., Tang, C.K.: Rethinking space-time networks with improved memory coverage for efficient video object segmentation. Adv. Neural Inf. Process. Syst. 34, 11781–11794 (2021)
Dou, Q., et al.: 3d deeply supervised network for automated segmentation of volumetric medical images. Med. Image Anal. 41, 40–54 (2017)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth international Conference on Artificial Intelligence and Statistics, pp. 249–256. JMLR Workshop and Conference Proceedings (2010)
Heller, N., et al.: The kits19 challenge data: 300 kidney tumor cases with clinical context, CT semantic segmentations, and surgical outcomes. arXiv preprint arXiv:1904.00445 (2019)
Heo, Y., Koh, Y.J., Kim, C.S.: Guided interactive video object segmentation using reliability-based attention maps. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7322–7330 (2021)
Isensee, F., et al.: nnU-net: Self-adapting framework for u-net-based medical image segmentation. arXiv preprint arXiv:1809.10486 (2018)
Lee, K., Zung, J., Li, P., Jain, V., Seung, H.S.: Superhuman accuracy on the snemi3d connectomics challenge. arXiv preprint arXiv:1706.00120 (2017)
Li, W., et al.: Interactive medical image segmentation with self-adaptive confidence calibration. arXiv preprint arXiv:2111.07716 (2021)
Liao, X., et al.: Iteratively-refined interactive 3d medical image segmentation with multi-agent reinforcement learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9394–9402 (2020)
Ma, C., et al.: Boundary-aware supervoxel-level iteratively refined interactive 3d image segmentation with multi-agent reinforcement learning. IEEE Trans. Med. Imaging 40(10), 2563–2574 (2020)
Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D vision (3DV), pp. 565–571. IEEE (2016)
Oh, S.W., Lee, J.Y., Xu, N., Kim, S.J.: Video object segmentation using space-time memory networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9226–9235 (2019)
Simpson, A.L., et al.: A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063 (2019)
Wang, G., et al.: Deepigeos: a deep interactive geodesic framework for medical image segmentation. IEEE Trans. Pattern Anal. Mach. intell. 41(7), 1559–1572 (2018)
Yin, Z., Zheng, J., Luo, W., Qian, S., Zhang, H., Gao, S.: Learning to recommend frame for interactive video object segmentation in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15445–15454 (2021)
Zhou, T., Li, L., Bredell, G., Li, J., Konukoglu, E.: Quality-aware memory network for interactive volumetric image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 560–570. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_52
Acknowledgements
This work was supported by Chinese Key-Area Research and Development Program of Guangdong Province (2020B0101350001) and the Guangdong Provincial Key Laboratory of Big Data Computing, The Chinese University of Hong Kong, Shenzhen. It was also supported by NSFC-61931024 and Shenzhen Sustainable Development Project(KCXFZ20201221173008022). We thank the ITSO in CUHKSZ for their High-Performance Computing Services.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shi, L., Zhang, X., Liu, Y., Han, X. (2022). A Hybrid Propagation Network for Interactive Volumetric Image Segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13434. Springer, Cham. https://doi.org/10.1007/978-3-031-16440-8_64
Download citation
DOI: https://doi.org/10.1007/978-3-031-16440-8_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16439-2
Online ISBN: 978-3-031-16440-8
eBook Packages: Computer ScienceComputer Science (R0)
