Abstract
Deep learning with Convolutional Neural Network (CNN) has exhibited high diagnostic performance for lesion characterization. However, it is still challenging to train powerful deep learning systems for lesion characterization, because there are often limited samples in different malignancy types and there exist considerable variabilities across images from multiple scanners in clinical practice. In this work, we propose a similarity steered generative adversarial network (SSGAN) coupled with pre-train and adaptive fine-turning of data from multiple scanners for lesion characterization. Specifically, SSGAN is based on adding a similarity discriminative measure in the conventional generative adversarial network to effectively generate more discrepant samples, while the adaptive fine-tune strategy is adopted to optimally make decisions on whether to use the pre-train layers or the fine-tune layers. Experimental results of pathologically confirmed malignancy of clinical hepatocellular carcinoma (HCCs) with MR images acquired by different scanners (GE, Philips and Siemens) demonstrate several intriguing characteristics of the proposed end-to-end framework for malignancy characterization of HCC as follows: (1) The proposed SSGAN remarkably improves the performance of lesion characterization and outperforms several recently proposed methods. (2) The adaptive fine-tuning combined with the proposed SSGAN can further improve the performance of lesion characterization in the context of limited data. (3) Clinical images acquired by one MR scanner for pre-train can be used to improve the characterization performance of images acquired by another MR scanner, outperforming the pre-train with ImageNet.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jonas, S., Bechstein, W.O., Steinmuller, T., et al.: Vascular invasion and histopathologic grading determine outcome after liver transplantation for hepatocellular carcinoma in cirrhosis. Hepatology 33(5), 1080–1086 (2001)
Miles, K.A., Ganeshan, B., Griffiths, M.R., Young, R.C., Chatwin, C.R.: Colorectal cancer: texture analysis of portal phase hepatic CT Images as a potential marker of survival. Radiology 250(2), 444–452 (2009)
Litjens, G., Kooi, T., Bejnordi, B.E., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42(7), 60–88 (2017)
Parmar, C., Barry, J.D., Hosny, A., et al.: Data analysis strategies in medical imaging. Clin. Cancer Res. 24(15), 3492–3499 (2018)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: NIPS (2014)
Frid-adar, M., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Synthetic data augmentation using GAN for improved liver lesion classification. In: IEEE 15th International Symposium on Biomedical Imaging, pp. 289–293(2018)
Salimans, T., Goodfellow I., Zaremba, W., et al.: Improved techniques for training GANs. In: NIPS (2016)
Shams, S., Platania, R., Kim, J., Park, S.: Deep generative cancer screening and diagnosis. In: MICCAI, pp. 859–867 (2018)
Yasaka, K., Akai, H., Kunimatsu, A., Kiryu, S., Abe, O.: Deep learning with convolutional neural network in radiology. Japan. J. Radiol. 36, 257–272 (2018)
Carneiro, G., Nascimento, J., Bradley, A.P.: Unregistered multiview mammogram analysis with pre-trained deep learning models. In: MICCAI, pp. 652–660 (2015)
Shermin, T., Murshed, M.M., Lu, G., et al.: An efficient transfer learning technique by using final fully-connected layer output features of deep networks. \({\rm arXiv:}\) Computer Vision and Pattern Recognition (2018)
Guo, Y., Shi, H., Humar, A., et al.: SpotTune: transfer learning through adaptive fine-tuning. arXiv:1811.08737v1 [cs CV] (2018)
Maddison, C.J., Mnih, A., Teh, Y.W.: The concrete distribution: a continuous relaxation of discrete random variables. arXiv: 1801.06519 (2016)
Jang, E., Gu, S., Poole, B.: Categorical reparameterization with gumbel-softmax. arXiv:1611.01144v5 [stat. ML] (2017)
Acknowledgment
This research is supported by the grant from National Natural Science Foundation of China (NSFC: 81771920).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ju, H., Jian, W., Cen, X., Wang, G., Zhou, W. (2019). Similarity Steered Generative Adversarial Network and Adaptive Transfer Learning for Malignancy Characterization of Hepatocellualr Carcinoma. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11767. Springer, Cham. https://doi.org/10.1007/978-3-030-32251-9_62
Download citation
DOI: https://doi.org/10.1007/978-3-030-32251-9_62
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32250-2
Online ISBN: 978-3-030-32251-9
eBook Packages: Computer ScienceComputer Science (R0)