Abstract
Airway semantic segmentation, which refers to segmenting airway from background and dividing it into anatomical segments, provides clinically valuable information for lung lobe analysis, pulmonary lesion localization, and comparison between different patients. It is technically challenging due to the complicated tree-like structure, individual variations, and severe class imbalance. We propose a structure-aware graph-based network (SGNet) for airway semantic segmentation directly from chest CT scans. The proposed framework consists of a feature extractor combining a multi-task U-Net with a structure-aware GCN, and an inference module comprised of two convolutional layers. The multi-task U-Net is trained to regress bifurcation landmark heatmaps, binary and semantic segmentation maps simultaneously, providing initial predictions for graph construction. By introducing irregular edges connecting voxels with the sampled points around corresponding bifurcation landmarks, the two-layer GCN incorporates the structural prior explicitly. Experiments on both public and private datasets demonstrate that the SGNet achieves superior and robust performance, even on subjects affected by severe pulmonary diseases.
This work was supported in part by the National Natural Science Foundation of China under Grants 82071921.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Annotations are available at https://cloud.tsinghua.edu.cn/d/43bbc05fb9714f71a56f/
References
van Ginneken, B., Baggerman, W., van Rikxoort, E.M.: Robust segmentation and anatomical labeling of the airway tree from thoracic CT scans. In: Metaxas, D., Axel, L., Fichtinger, G., Székely, G. (eds.) MICCAI 2008. LNCS, vol. 5241, pp. 219–226. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85988-8_27
Tschirren, J., McLennan, G., Palágyi, K., et al.: Matching and anatomical labeling of human airway tree. IEEE T-MI 24(12), 1540–1547 (2005)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Qin, Y., et al.: AirwayNet: a voxel-connectivity aware approach for accurate airway segmentation using convolutional neural networks. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 212–220. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_24
Qin, Y., Gu, Y., Zheng, H., et al.: AirwayNet-SE: a simple-yet-effective approach to improve airway segmentation using context scale fusion. In: ISBI (2020)
Qin, Y., et al.: Learning bronchiole-sensitive airway segmentation CNNs by feature recalibration and attention distillation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 221–231. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_22
Charbonnier, J.P., Van Rikxoort, E.M., Setio, A.A., et al.: Improving airway segmentation in computed tomography using leak detection with convolutional networks. MedIA 36, 52–60 (2017)
Meng, Q., Roth, H.R., Kitasaka, T., Oda, M., Ueno, J., Mori, K.: Tracking and segmentation of the airways in chest CT using a fully convolutional network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 198–207. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_23
Yun, J., Park, J., Yu, D., et al.: Improvement of fully automated airway segmentation on volumetric computed tomographic images using a 2.5 dimensional convolutional neural net. MedIA 51, 13–20 (2019)
Wang, C., et al.: Tubular structure segmentation using spatial fully connected network with radial distance loss for 3D medical images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 348–356. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_39
Zhao, T., Yin, Z., Wang, J., Gao, D., Chen, Y., Mao, Y.: Bronchus segmentation and classification by neural networks and linear programming. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 230–239. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_26
Gu, S., Wang, Z., Siegfried, J.M., et al.: Automated lobe-based airway labeling. Int. J. Biomed. Imaging 2012 (2012)
Feragen, A.: A hierarchical scheme for geodesic anatomical labeling of airway trees. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7512, pp. 147–155. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33454-2_19
Wu, D., et al.: Automated anatomical labeling of coronary arteries via bidirectional tree LSTMs. Int. J. Comput. Assist. Radiol. Surg. 14(2), 271–280 (2018). https://doi.org/10.1007/s11548-018-1884-6
Yang, H., Zhen, X., Chi, Y., et al.: CPR-GCN: conditional partial-residual graph convolutional network in automated anatomical labeling of coronary arteries. In: CVPR (2020)
Cao, Q., et al.: Automatic identification of coronary tree anatomy in coronary computed tomography angiography. Int. J. Cardiovasc. Imaging 33(11), 1809–1819 (2017). https://doi.org/10.1007/s10554-017-1169-0
Robben, D., Türetken, E., Sunaert, S., et al.: Simultaneous segmentation and anatomical labeling of the cerebral vasculature. MedIA 32, 201–215 (2016)
Matsuzaki, T., Oda, M., Kitasaka, T., et al.: Automated anatomical labeling of abdominal arteries and hepatic portal system extracted from abdominal CT volumes. MedIA 20(1), 152–161 (2015)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)
Selvan, R., Kipf, T., Welling, M., et al.: Extraction of airways using graph neural networks. In: MIDL (2018)
Selvan, R., Kipf, T., Welling, M., et al.: Graph refinement based airway extraction using mean-field networks and graph neural networks. MedIA 64, 101751 (2020)
Shin, S.Y., Lee, S., Yun, I.D., et al.: Deep vessel segmentation by learning graphical connectivity. MedIA 58, 101556 (2019)
Juarez, A.G.U., Selvan, R., Saghir, Z., de Bruijne, M.: A joint 3D unet-graph neural network-based method for airway segmentation from chest CTs. In: MLMI (2019)
Yao, L., Jiang, P., Xue, Z., et al.: Graph convolutional network based point cloud for head and neck vessel labeling. In: MLMI (2020)
Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_27
Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 3DV (2016)
Armato, I.I.I., Samuel, G., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys. 38(2), 915–931 (2011)
Lo, P., Ginneken, B.V., Reinhardt, J.M., et al.: Extraction of airways from CT (EXACT’09). IEEE T-MI 31(11), 2093–2107 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Tan, Z., Feng, J., Zhou, J. (2021). SGNet: Structure-Aware Graph-Based Network for Airway Semantic Segmentation. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12901. Springer, Cham. https://doi.org/10.1007/978-3-030-87193-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-87193-2_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87192-5
Online ISBN: 978-3-030-87193-2
eBook Packages: Computer ScienceComputer Science (R0)
