Abstract
Semi-supervised learning (SSL) algorithms have attracted much attentions in medical image segmentation due to challenge in acquiring pixel-wise annotations by using unlabeled data. However, most of existing SSLs neglected the geometric shape constraint in object, leading to unsatisfactory boundary and non-smooth of object. In this paper, we propose a shape-aware semi-supervised 3D medical image segmentation network, named 3D Graph-S2Net, which incorporates the flexible shape information and learns duality constraints between semantics and geometrics in the graph domain. Specifically, our method consists of two parts: a multi-task learning network (3D S2Net) and a graph-based cross-task module (3D BGCM). The 3D S2Net improves the existing self-ensembling model (i.e., Mean-Teacher model) by adding a signed distance map (SDM) prediction task, which encodes richer features of object shape and surface. Moreover, the 3D BGCM explores the co-occurrence relations between the semantics segmentation and SDM prediction task, so that the network learns stronger semantic and geometric correspondences from both labeled and unlabeled data. Experimental results on the Atrial Segmentation Challenge confirm that our 3D Graph-S2Net outperforms the state-of-the-arts in semi-supervised segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lu, F., et al.: Automatic 3D liver location and segmentation via convolutional neural network and graph cut. Int. J. Comput. Assist. Radiol. Surg. 12(2), 171–182 (2016). https://doi.org/10.1007/s11548-016-1467-3
Bai, W., et al.: Semi-supervised learning for network-based cardiac MR image segmentation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 253–260. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_29
Xia, Y., et al.: 3D semi-supervised learning with uncertainty - aware multi-view co-training. arXiv:1811.12506 (2018)
Zhou, Y., et al.: Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training. In: IEEE Winter Conference on Applications of Computer Vision (2019)
Nie, D., Gao, Y., Wang, L., Shen, D.: ASDNet: attention based semi-supervised deep networks for medical image segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 370–378. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_43
Zheng, H., et al.: Semi-supervised segmentation of liver using adversarial learning with deep atlas prior. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 148–156. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_17
Huang, H., et al.: Medical image segmentation with deep atlas prior. IEEE Trans. Med. Imaging (2021)
Li, S., Zhang, C., He, X.: Shape-aware semi-supervised 3D semantic segmentation for medical images. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 552–561. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_54
Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. In: 5th International Conference on Learning Representations, Toulon, France (2017)
Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. NIPS (2017)
Cui, W., et al.: Semi-supervised brain lesion segmentation with an adapted mean teacher model. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 554–565. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_43
Yu, L., Wang, S., Li, X., Fu, C.-W., Heng, P.-A.: Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: Shen, Dinggang, et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 605–613. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_67
Dangi, S., Linte, C.A., Yaniv, Z.: A distance map regularized CNN for cardiac cine MR image segmentation. Med. Phys. 46, 5637–5651 (2019)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (2017)
Li, Y., Abhinav, G.: Beyond grids: Learning graph representations for visual recognition. Adv. Neural Inf. Process. Syst. 31, 9225–9235 (2018)
Chen, Y., et al.: Graph-based global reasoning networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)
Huang, H., et al.: Graph-based pyramid global context reasoning with a saliency-aware projection for COVID-19 lung infections segmentation. In: ICASSP (2021)
Huang, S., et al.: Referring image segmentation via cross-modal progressive comprehension. In: Computer Vision and Pattern Recognition (2020)
Shin, S., Lee, S., Yun, I., Lee, K.: Deep vessel segmentation by learning graphical connectivity. Med. image Anal. 58, 101556 (2019)
Li, X., et al.: Spatial pyramid based graph reasoning for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)
Zhang, L., Li, X., Arnab, A., Yang, K., Tong, Y., Torr, P.: Dual graph convolutional network for semantic segmentation. arXiv preprint arXiv:1909.06121 (2019)
Te, G., et al.: Edge-aware graph representation learning and reasoning for face parsing. arXiv preprint arXiv:2007.11240 (2020)
Wu, Y., Zhang, G., et al.: Bidirectional graph reasoning network for panoptic segmentation. In: Conference on Computer Vision and Pattern Recognition (2020)
Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 3DV (2016)
Velickovic, P., Cucurull, G., Casanova, A., et al.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Atrial Segmentation Challenge. http://atriaseg2018.cardiacatlas.org/
Li, X., et al.: Semi-supervised skin lesion segmentation via transformation consistent self-ensembling model. arXiv preprint arXiv:1808.03887 (2018)
Vu, T.-H., et al.: Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: CVRP (2019)
Ouali, Y., et al.: Semi-supervised semantic segmentation with cross-consistency training. In: CVPR (2020)
Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: CVPR (2019)
Xue, Y., et al.: Shape-aware organ segmentation by predicting signed distance maps. arXiv preprint arXiv:1912.03849 (2019)
Perera, S., Barnes, N., He, X., Izadi, S., Kohli, P., Glocker, B.: Motion segmentation of truncated signed distance function based volumetric surfaces. In: WACV (2015)
Huang, H., et al.: UNet 3+: a full-scale connected UNet for medical image segmentation. In: ICASSP (2020)
Chen, L.-C., et al.: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Acknowledgement
This work was supported in part by Major Scientific Research Project of Zhejiang Lab under Grant No. 2020ND8AD01, and in part by Grant-in Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under Grant No. 20KK0234, No. 21H03470 and No. 20K21821.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, H. et al. (2021). 3D Graph-S2Net: Shape-Aware Self-ensembling Network for Semi-supervised Segmentation with Bilateral Graph Convolution. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12902. Springer, Cham. https://doi.org/10.1007/978-3-030-87196-3_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-87196-3_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87195-6
Online ISBN: 978-3-030-87196-3
eBook Packages: Computer ScienceComputer Science (R0)
