Abstract
The segmentation of renal tumor, quantification of tumor indices (i.e., the center point coordinates, diameter, circumference, and cross-sectional area) and uncertainty estimation of segmentation are the key processes for clinical tumor disease diagnosis. However, these tasks have been studied independently so far. Because segmentation and quantification tasks have different optimization types, representing two different tasks as a unified optimization framework is a severe challenge. In this paper, we propose a unified framework (i.e., Mt-UcGAN: multi-task uncertainty-constrained generative adversarial network) for joint segmentation, quantification, and uncertainty estimation of renal tumors on CT. Mt-UcGAN includes a multitasking integrated generator (MtIG) and an uncertainty-constrained discriminator (UcD). MtIG achieves multi-task joint learning by novelly merging skip connections and Monte Carlo sampling. UCD guides the learning of segmentation and quantification networks by innovatively feeding prior information with high uncertainty constraints. Mt-UcGAN effectively corrects tumor prediction errors and improves network performance through continuous adversarial learning and alternate training. Experiments are performed on CT of 113 renal tumor patients. The dice coefficient of Mt-UcGAN is 92.1%, and the \(R^2\) coefficient of tumor circumference is 0.9513. The results show that this method has great potential to be extended to other medical image analysis tasks and clinical application value.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mehrazin, R., et al.: Impact of tumour morphology on renal function decline after partial nephrectomy. BJU Int. 111(8), E374–E382 (2013)
Greene, F.L., et al.: AJCC Cancer Staging Handbook: TNM Classification of Malignant Tumors. Springer Science & Business Media, New York (2002)
Spaliviero, M., et al.: Interobserver variability of RENAL, PADUA, and centrality index nephrometry score systems. World J. Urol. 33(6), 853–858 (2015)
Kutikov, A., Uzzo, R.G.: The renal nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth. J. Urol. 182(3), 844–853 (2009)
Ficarra, V., et al.: Preoperative aspects and dimensions used for an anatomical (PADUA) classification of renal tumours in patients who are candidates for nephron-sparing surgery. Eur. Urol. 56(5), 786–793 (2009)
Taha, A., Lo, P., Li, J., Zhao, T.: Kid-Net: convolution networks for kidney vessels segmentation from CT-volumes. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 463–471. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_53
Yang, G., et al.: Automatic segmentation of kidney and renal tumor in CT images based on 3d fully convolutional neural network with pyramid pooling module. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 3790–3795. IEEE (2018)
Edge, S.B., et al.: AJCC Cancer Staging Manual, vol. 649. Springer, New York (2010)
Afshin, M., Ayed, I.B., Islam, A., Goela, A., Peters, T.M., Li, S.: Global assessment of cardiac function using image statistics in MRI. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7511, pp. 535–543. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33418-4_66
Zhen, X., Wang, Z., Yu, M., Li, S.: Supervised descriptor learning for multi-output regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1211–1218 (2015)
Zhen, X., Zhang, H., Islam, A., Bhaduri, M., Chan, I., Li, S.: Direct and simultaneous estimation of cardiac four chamber volumes by multioutput sparse regression. Med. Image Anal. 36, 184–196 (2017)
Wu, H., Bailey, C., Rasoulinejad, P., Li, S.: Automatic landmark estimation for adolescent idiopathic scoliosis assessment using BoostNet. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 127–135. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_15
Sun, H., Zhen, X., Bailey, C., Rasoulinejad, P., Yin, Y., Li, S.: Direct estimation of spinal cobb angles by structured multi-output regression. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 529–540. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59050-9_42
Zhen, X., Yu, M., Islam, A., Bhaduri, M., Chan, I., Li, S.: Descriptor learning via supervised manifold regularization for multioutput regression. IEEE Trans. Neural Netw. Learn. Syst. 28(9), 2035–2047 (2016)
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., Jemal, A.: Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 68(6), 394–424 (2018)
Xu, C., Howey, J., Ohorodnyk, P., Roth, M., Zhang, H., Li, S.: Segmentation and quantification of infarction without contrast agents via spatio temporal generative adversarial learning. Med. Image Anal., 101568 (2019)
Luo, G., et al.: Commensal correlation network between segmentation and direct area estimation for bi-ventricle quantification. Med. Image Anal., 101591 (2019)
Ruan, Y., et al.: MB-FSGAN: joint segmentation and quantification of kidney tumor on CT by the multi-branch feature sharing generative adversarial network. Med. Image Anal. (2020)
Jungo, A., Reyes, M.: Assessing reliability and challenges of uncertainty estimations for medical image segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 48–56. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_6
Raghu, M., et al.: Direct uncertainty prediction for medical second opinions. arXiv preprint arXiv:1807.01771 (2018)
Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T.: Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing 338, 34–45 (2019)
Xia, K.J., Yin, H.S., Zhang, Y.D.: Deep semantic segmentation of kidney and space-occupying lesion area based on SCNN and ResNet models combined with SIFT-flow algorithm. J. Med. Syst. 43(1), 2 (2019)
Yin, K., Liu, C., Bardis, M., Martin, J., Liu, H., Ushinsky, A., Glavis-Bloom, J., Chantaduly, C., Chow, D.S., Houshyar, R., et al.: Deep learning segmentation of kidneys with renal cell carcinoma. J. Clin. Oncol. 37, e16098–e16098 (2019)
Yu, Q., Shi, Y., Sun, J., Gao, Y., Zhu, J., Dai, Y.: Crossbar-Net: a novel convolutional neural network for kidney tumor segmentation in CT images. IEEE Trans. Image Process. 28(8), 4060–4074 (2019)
Acknowledgments
This work was funded by the National Natural Science Foundation of China (61971271), the Taishan Scholars Project of Shandong Province (Tsqn20161023) and the Primary Research and Development Plan of Shandong Province (No. 2018GGX101018, No. 2019QYTPY020).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ruan, Y. et al. (2020). Mt-UcGAN: Multi-task Uncertainty-Constrained GAN for Joint Segmentation, Quantification and Uncertainty Estimation of Renal Tumors on CT. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12264. Springer, Cham. https://doi.org/10.1007/978-3-030-59719-1_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-59719-1_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59718-4
Online ISBN: 978-3-030-59719-1
eBook Packages: Computer ScienceComputer Science (R0)
