Abstract
Modern multi-site neuroimaging studies are known to be biased by significant site effects observed in imaging data and their derived structural and functional features. Although many statistical models and deep learning methods have been proposed to eliminate the site effects while maintaining biological characteristics, they have two major drawbacks. First, statistical models are applicable for harmonizing regional-level data but are inherently not suitable to represent the complex non-linear mapping of vertex-wise cortical property maps. Second, existing deep learning methods can only harmonize data between two sites, which are practically less useful in multi-site data harmonization scenario and also ignore the rich information in the whole dataset. To address these issues, we develop a novel, flexible deep learning method to harmonize multi-site cortical surface property maps. Specifically, to detect and remove site effects, we employ a surface-based autoencoder and decompose the encoded cortical features into site-related and site-unrelated components and use an adversarial strategy to encourage the disentanglement. Then decoding the site-unrelated features with other site features can generate mappings across different sites. To learn more controllable and meaningful mappings, we enforce the cycle consistency between forward and backward mappings. Our method can thus efficiently learn rich information from the whole dataset and generate realistic harmonized surface maps at the target site. Experiments on harmonizing infant cortical thickness maps of 2,342 scans from four sites with different scanners and imaging protocols validate the superior performance of our method on both site effects removal and biological variability preservation compared to other methods. To the best of our knowledge, this is the largest validation of different methods on infant cortical data harmonization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bzdok, D.: Classical statistics and statistical learning in imaging neuroscience. Front. Neurosci. 11, 543 (2017)
Cackowski, S., Barbier, E.L., Dojat, M., Christen, T.: ImUnity: a generalizable VAE-GAN solution for multicenter MR image harmonization. Med. Image Anal. 88, 102799 (2023)
Desikan, R.S., et al.: An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31(3), 968–980 (2006)
Dewey, B.E., et al.: DeepHarmony: a deep learning approach to contrast harmonization across scanner changes. Magn. Reson. Imaging 64, 160–170 (2019)
Ding, Z., et al.: Guided variational autoencoder for disentanglement learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7920–7929 (2020)
Fischl, B.: FreeSurfer. Neuroimage 62(2), 774–781 (2012)
Fortin, J.P., et al.: Harmonization of cortical thickness measurements across scanners and sites. Neuroimage 167, 104–120 (2018)
Frisoni, G.B., Fox, N.C., Jack, C.R., Jr., Scheltens, P., Thompson, P.M.: The clinical use of structural MRI in Alzheimer disease. Nat. Rev. Neurol. 6(2), 67–77 (2010)
Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 1–35 (2016)
Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)
Guan, H., Liu, M.: DomainATM: domain adaptation toolbox for medical data analysis. NeuroImage 268, 119863 (2023)
Hazlett, H.C., et al.: Early brain development in infants at high risk for autism spectrum disorder. Nature 542(7641), 348–351 (2017)
Howell, B.R., et al.: The UNC/UMN baby connectome project (BCP): an overview of the study design and protocol development. Neuroimage 185, 891–905 (2019)
Hu, F., et al.: Image harmonization: a review of statistical and deep learning methods for removing batch effects and evaluation metrics for effective harmonization. NeuroImage 274, 120125 (2023)
Ij, H.: Statistics versus machine learning. Nat. Meth. 15(4), 233 (2018)
Kazeminia, S., et al.: GANs for medical image analysis. Artif. Intell. Med. 109, 101938 (2020)
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., Frey, B.: Adversarial autoencoders. arXiv preprint arXiv:1511.05644 (2015)
Moyer, D., Ver Steeg, G., Tax, C.M., Thompson, P.M.: Scanner invariant representations for diffusion MRI harmonization. Magn. Reson. Med. 84(4), 2174–2189 (2020)
Pomponio, R.: Harmonization of large MRI datasets for the analysis of brain imaging patterns throughout the lifespan. Neuroimage 208, 116450 (2020)
Solanes, A., et al.: Removing the effects of the site in brain imaging machine-learning-measurement and extendable benchmark. Neuroimage 265, 119800 (2023)
Torbati, M.E., et al.: A multi-scanner neuroimaging data harmonization using ravel and combat. Neuroimage 245, 118703 (2021)
Wang, F., et al.: Developmental topography of cortical thickness during infancy. Proc. Natl. Acad. Sci. 116(32), 15855–15860 (2019)
Wang, L., Wu, Z., Chen, L., Sun, Y., Lin, W., Li, G.: iBEAT V2. 0: a multisite-applicable, deep learning-based pipeline for infant cerebral cortical surface reconstruction. Nat. Protoc. 18, 1488–1509 (2023)
Xia, K.: Genetic influences on longitudinal trajectories of cortical thickness and surface area during the first 2 years of life. Cereb. Cortex 32(2), 367–379 (2022)
Xia, Y., Shi, Y.: Personalized DMRI harmonization on cortical surface. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13436, pp. 717–725. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16446-0_68
Yamashita, A., et al.: Harmonization of resting-state functional MRI data across multiple imaging sites via the separation of site differences into sampling bias and measurement bias. PLoS Biol. 17(4), e3000042 (2019)
Zhao, F., et al.: Spherical deformable U-Net: application to cortical surface parcellation and development prediction. IEEE Trans. Med. Imaging 40(4), 1217–1228 (2021)
Zhao, F., et al.: Harmonization of infant cortical thickness using surface-to-surface cycle-consistent adversarial networks. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 475–483. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_52
Zhao, F.: Spherical U-Net on cortical surfaces: methods and applications. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 855–866. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_67
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Zuo, L., et al.: Unsupervised MR harmonization by learning disentangled representations using information bottleneck theory. Neuroimage 243, 118569 (2021)
Acknowledgements
This work was supported in part by the National Institutes of Health (NIH) under Grants MH116225, MH117943, MH123202, AG075582, and NS128534.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, F. et al. (2023). Disentangling Site Effects with Cycle-Consistent Adversarial Autoencoder for Multi-site Cortical Data Harmonization. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14227. Springer, Cham. https://doi.org/10.1007/978-3-031-43993-3_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-43993-3_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43992-6
Online ISBN: 978-3-031-43993-3
eBook Packages: Computer ScienceComputer Science (R0)
