Abstract
Prediction of microvascular invasion (MVI) in hepatocellular carcinoma (HCC) has important clinical value for treatment decisions and prognosis. Diffusion-weighted imaging (DWI) intravoxel incoherent motion (IVIM) models have been used to predict MVI in HCC. However, the parameter fitting of the IVIM model based on the typical nonlinear least squares method has a large amount of computation, and its accuracy is disturbed by noise. In addition, the performance of characterizing tumor characteristics based on the feature of IVIM parameter values is limited. In order to overcome the above difficulties, we proposed a novel multi-task deep learning network based on transformer to simultaneously conduct IVIM parameter model fitting and MVI prediction. Specifically, we utilize the transformer’s powerful long-distance feature modeling ability to encode deep features of different tasks, and then generalize self-attention to cross-attention to match features that are beneficial to each task. In addition, inspired by the work of Compact Convolutional Transformer (CCT), we design the multi-task learning network based on CCT to enable the transformer to work in the small dataset of medical images. Experimental results of clinical HCC with IVIM data show that the proposed transformer based multi-task learning method is better than the current multi-task learning methods based on attention. Moreover, the performance of MVI prediction and IVIM model fitting based on multi-task learning is better than those of single-task learning methods. Finally, IVIM model fitting facilitates the performance of IVIM to characterize MVI, providing an effective tool for clinical tumor characterization.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barbieri, S., Gurney-Champion, O.J., Klaassen, R., Thoeny, H.C.: Deep learning how to fit an intravoxel incoherent motion model to diffusion-weighted MRI. Magn. Reson. Med. 83(1), 312–321 (2020)
Baxter, J.: A Bayesian/information theoretic model of learning to learn via multiple task sampling. Mach. Learn. 28(1), 7–39 (1997)
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Houlsby, N.: An image is worth 16x16 words: transformers for image recognition at scale (2020)
Erstad, D.J., Tanabe, K.K.: Prognostic and therapeutic implications of microvascular invasion in hepatocellular carcinoma. Ann. Surg. Oncol. 26(5), 1474–1493 (2019)
Hassani, A., Walton, S., Shah, N., Abuduweili, A., Li, J., Shi, H.: Escaping the big data paradigm with compact transformers (2021)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. IEEE (2016)
Hernando, D., Zhang, Y., Pirasteh, A.: Quantitative diffusion MRI of the abdomen and pelvis. Med. Phys. (2021)
Iima, M., Le Bihan, D.: Clinical intravoxel incoherent motion and diffusion MR imaging: past, present, and future. Radiology 278(1), 13–32 (2016)
Lanzarone, E., Mastropietro, A., Scalco, E., Vidiri, A., Rizzo, G.: A novel Bayesian approach with conditional autoregressive specification for intravoxel incoherent motion diffusion-weighted MRI. NMR Biomed. 33(3), e4201 (2020)
Le Bihan, D., Breton, E., Lallemand, D., Aubin, M., Vignaud, J., Laval-Jeantet, M.: Separation of diffusion and perfusion in intravoxel incoherent motion MR imaging. Radiology 168(2), 497–505 (1988)
Li, H., et al.: Preoperative histogram analysis of intravoxel incoherent motion (IVIM) for predicting microvascular invasion in patients with single hepatocellular carcinoma. Eur. J. Radiol. 105, 65–71 (2018)
Liu, S., Johns, E., Davison, A.J.: End-to-end multi-task learning with attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1871–1880 (2019)
Lyu, K., Li, Y., Zhang, Z.: Attention-aware multi-task convolutional neural networks. IEEE Trans. Image Process. PP(99), 1–1 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sung, H., et al.: Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 71(3), 209–249 (2021)
Ulas, C., et al.: Convolutional neural networks for direct inference of pharmacokinetic parameters: application to stroke dynamic contrast-enhanced MRI. Front. Neurol. 1147 (2019)
Vaswani, A., et al.: Attention is all you need. arXiv (2017)
Vasylechko, S.D., Warfield, S.K., Afacan, O., Kurugol, S.: Self-supervised IVIM DWI parameter estimation with a physics based forward model. Magn. Reson. Med. 87(2), 904–914 (2022)
Wang, A.G., et al.: Prediction of microvascular invasion of hepatocellular carcinoma based on preoperative diffusion-weighted MR using deep learning. Acad. Radiol. (2020)
Wei, Y., et al.: IVIM improves preoperative assessment of microvascular invasion in HCC. Eur. Radiol. 29(10), 5403–5414 (2019)
Zeng, Q., Liu, B., Xu, Y., Zhou, W.: An attention-based deep learning model for predicting microvascular invasion of hepatocellular carcinoma using an intra-voxel incoherent motion model of diffusion-weighted magnetic resonance imaging. Phys. Med. Biol. 66(18), 185019 (2021)
Zhao, W., et al.: Preoperative prediction of microvascular invasion of hepatocellular carcinoma with IVIM diffusion-weighted MR imaging and GD-EOB-DTPA-enhanced MR imaging. PLoS ONE 13(5), e0197488 (2018)
Acknowledgements
This work is supported by the grant from National Nature Science Foundation of China (No. 81771920).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, H., Liu, B., Zhang, L., Xu, Y., Zhou, W. (2022). Transformer Based Multi-task Deep Learning with Intravoxel Incoherent Motion Model Fitting for Microvascular Invasion Prediction of Hepatocellular Carcinoma. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13437. Springer, Cham. https://doi.org/10.1007/978-3-031-16449-1_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-16449-1_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16448-4
Online ISBN: 978-3-031-16449-1
eBook Packages: Computer ScienceComputer Science (R0)