Abstract
Automated segmentation methods for image segmentation have the potential to support diagnosis and prognosis using medical images in clinical practice. To achieve the goal of HEad and neCK tumOR (HECKTOR) segmentation and outcome survival prediction in PET/CT images in the MICCAI 2021 challenge, we proposed a novel framework to segment head and neck tumors by leveraging multi-modal imaging using a cross-attention module based on a dual-path and ensemble modeling with majority voting. In addition, we expanded our task to survival analysis using a random forest survival model to predict the prognosis of tumors using clinical information and segmented tumor volume. Our segmentation model achieved a Dice coefficient and Hausdorff distance of 0.7367 and 3.2700, respectively. Our survival model showed a concordance index (C-index) of 0.6459.
Keywords
J. Lee, J. Kang and E. Y. Shin—Equal contribution.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Andrearczyk, V., et al.: Overview of the HECKTOR challenge at MICCAI 2021: automatic head and neck tumor segmentation and outcome prediction in PET/CT images. In: Andrearczyk, V., Oreiller, V., Hatt, M., Depeursinge, A. (eds.) HECKTOR 2021. LNCS, vol. 13209, pp. 1–37. Springer, Cham (2022)
Baltrušaitis, T., Ahuja, C., Morency, L.P.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 423–443 (2018)
Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 539–546. IEEE (2005)
Cook, R.D.: Detection of Influential Observation in Linear Regression. Technometrics 19(1), 15–18 (1977)
Edge, S.B., Byrd, D.R., Carducci, M.A., Compton, C.C., Fritz, A., Greene, F., et al.: AJCC Cancer Staging Manual, vol. 7. Springer, New York (2010)
Harrell, F.E., Jr., Lee, K.L., Mark, D.B.: Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med. 15(4), 361–387 (1996)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Howlader, N., et al.: SEER Cancer Statistics Review, 1975–2018. National Cancer Institute (2021)
Iantsen, A., Visvikis, D., Hatt, M.: Squeeze-and-excitation normalization for automated delineation of head and neck primary tumors in combined PET and CT images. In: Andrearczyk, V., Oreiller, V., Depeursinge, A. (eds.) HECKTOR 2020. LNCS, vol. 12603, pp. 37–43. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67194-5_4
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)
Ishwaran, H., Kogalur, U.B., Blackstone, E.H., Lauer, M.S.: Random survival forests. Ann. Appl. Stat. 2(3), 841–860 (2008)
Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980 (2014)
Massa, S.T., Osazuwa-Peters, N., Boakye, E.A., Walker, R.J., Ward, G.M.: Comparison of the Financial Burden of Survivors of Head and Neck Cancer with Other Cancer Survivors. JAMA Otolaryngol. Head Neck Surg. 145(3), 239–249 (2019)
Oreiller, V., et al.: Head and neck tumor segmentation in PET/CT: the HECKTOR challenge. Med. Image Anal., 102336 (2021)
Paszke, A., et al.: Pytorch: An Imperative Style, High-Performance Deep Learning Library. Adv. Neural. Inf. Process. Syst. 32, 8026–8037 (2019)
Pérez-García, F., Sparks, R., Ourselin, S.: TorchIO: a Python Library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. Computer Methods and Programs in Biomedicine, p. 106236 (2021)
Pölsterl, S.: scikit-survival: a library for time-to-event analysis built on top of scikit-learn. J. Mach. Learn. Res. 21(212), 1–6 (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Seabold, S., Perktold, J.: Statsmodels: econometric and statistical modeling with python. In: Proceedings of the 9th Python in Science Conference, vol. 57, p. 61. Austin, TX (2010)
Siegel, R.L., Miller, K.D., Fuchs, H.E., Jemal, A.: Cancer statistics. CA: Cancer J. Clin. 71(1), 7–33 (2021). https://doi.org/10.3322/caac.21654
Siegel, R.L., Miller, K.D., Jemal, A.: Cancer statistics. CA: A Cancer J. Clinicians 66(1), 7–30 (2016)
Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 240–248. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_28
Vallieres, M., et al.: Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Sci. Rep. 7(1), 1–14 (2017)
Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)
Yeh, S.A.: Radiotherapy for head and neck cancer. In: Seminars in Plastic Surgery. vol. 24, pp. 127–136. Thieme Medical Publishers (2010)
Acknowledgements
This research was supported by the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI) and Korea Dementia Research Center (KDRC), funded by the Ministry of Health & Welfare and Ministry of Science and ICT, Republic of Korea (grant number: HU20C0315).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Lee, J., Kang, J., Shin, E.Y., Kim, R.E.Y., Lee, M. (2022). Dual-Path Connected CNN for Tumor Segmentation of Combined PET-CT Images and Application to Survival Risk Prediction. In: Andrearczyk, V., Oreiller, V., Hatt, M., Depeursinge, A. (eds) Head and Neck Tumor Segmentation and Outcome Prediction. HECKTOR 2021. Lecture Notes in Computer Science, vol 13209. Springer, Cham. https://doi.org/10.1007/978-3-030-98253-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-98253-9_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98252-2
Online ISBN: 978-3-030-98253-9
eBook Packages: Computer ScienceComputer Science (R0)