Skip to main content
SpringerLink
Log in
Book cover

Workshop on the Ethical and Philosophical Issues in Medical Imaging

International Workshop on Multimodal Learning for Clinical Decision Support

MICCAI Workshop on Topological Data Analysis for Biomedical Imaging

EPIMI 2022, ML-CDS 2022, TDA4BiomedicalImaging 2022: Ethical and Philosophical Issues in Medical Imaging, Multimodal Learning and Fusion Across Scales for Clinical Decision Support, and Topological Data Analysis for Biomedical Imaging pp 3–13Cite as

Data Poisoning Attack and Defenses in Connectome-Based Predictive Models

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13755))

Abstract

Connectome-based predictive models are widely used in the neuroimaging community and hold great clinical potential. Recent literature has focused on improving the accuracy and fairness of connectome-based models, while largely overlooking trustworthiness, defined as the robustness of a model to data manipulations. In this work, we investigate the idea of trustworthiness through backdoor data poisoning—a technique that manipulates a portion of the training data to encourage misclassification of a specific subset of testing data, while all other testing data remain unaffected. Furthermore, we demonstrate two defenses that mitigate, but do not completely prevent, the effects of data poisoning: randomized discretization and leave-one-site-out ensemble detection. Our findings suggest that trustworthiness in connectome-based predictive models needs to be carefully evaluated before any clinical applications and that defenses are necessary to ensure model outputs are trustworthy. Code is available at https://github.com/mattrosenblatt7/connectome_poisoning.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Casey, B.J., et al.: The adolescent brain cognitive development (ABCD) study: imaging acquisition across 21 sites. Dev. Cogn. Neurosci. 32, 43–54 (2018)

    Article  Google Scholar 

  2. Chen, X., Liu, C., Li, B., Lu, K., Song, D.: Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017)

  3. Cinà, A.E., et al.: Wild patterns reloaded: a survey of machine learning security against training data poisoning. arXiv preprint arXiv:2205.01992 (2022)

  4. Dadi, K., et al.: Alzheimer’s Disease Neuroimaging Initiative: benchmarking functional connectome-based predictive models for resting-state fMRI. Neuroimage 192, 115–134 (2019)

    Article  Google Scholar 

  5. Feng, Y., Ma, B., Zhang, J., Zhao, S., Xia, Y., Tao, D.: FIBA: frequency-Injection based backdoor attack in medical image analysis. arXiv preprint arXiv:2112.01148 (2021)

  6. Finlayson, S.G., Bowers, J.D., Ito, J., Zittrain, J.L., Beam, A.L., Kohane, I.S.: Adversarial attacks on medical machine learning. Science 363(6433), 1287–1289 (2019)

    Article  Google Scholar 

  7. Finlayson, S.G., Chung, H.W., Kohane, I.S., Beam, A.L.: Adversarial attacks against medical deep learning systems. arXiv preprint arXiv:1804.05296 (2018)

  8. Kumar, R.S.S., et al.: Adversarial machine learning - industry perspectives. In: IEEE Symposium on Security and Privacy Workshops (2020)

    Google Scholar 

  9. Marek, S., et al.: Towards reproducible Brain-Wide association studies. bioRxiv preprint bioRxiv:2020.08.21.257758 (2020)

    Google Scholar 

  10. Matsuo, Y., Takemoto, K.: Backdoor attacks to deep neural network-based system for COVID-19 detection from chest X-ray images. NATO Adv. Sci. Inst. Ser. E Appl. Sci. 11(20), 9556 (2021)

    Google Scholar 

  11. McNemar, Q.: Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12(2), 153–157 (1947)

    Article  Google Scholar 

  12. Nwadike, M., Miyawaki, T., Sarkar, E., Maniatakos, M., Shamout, F.: Explainability matters: backdoor attacks on medical imaging. arXiv preprint arXiv:2101.00008 (2020)

  13. Ortega, P.A., Figueroa, C.J., Ruz, G.A.: A medical claim Fraud/Abuse detection system based on data mining: a case study in Chile. In: Conference on Data Mining (2006)

    Google Scholar 

  14. Pedregosa, F., et al.: Others: scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  15. Pogue, J.M., Devereaux, P.J., Thorlund, K., Yusuf, S.: Central statistical monitoring: detecting fraud in clinical trials. Clin. Trials 10(2), 225–235 (2013)

    Article  Google Scholar 

  16. Rosenblatt, M., et al.: Can we trust machine learning in fMRI? Simple adversarial attacks break connectome-based predictive models (2021). OSF preprint https://doi.org/10.31219/osf.io/ptuwe

  17. Rudman, W.J., Eberhardt, J.S., 3rd., Pierce, W., Hart-Hester, S.: Healthcare fraud and abuse. Perspect. Health Inf. Manag. 6, 1g (2009)

    Google Scholar 

  18. Shafahi, A., Huang, W.R., Najibi, M., et al.: Poison frogs! targeted clean-label poisoning attacks on neural networks. In: Advances in Neural Information Processing Systems (2018)

    Google Scholar 

  19. Shen, X., Tokoglu, F., Papademetris, X., Constable, R.T.: Groupwise whole-brain parcellation from resting-state fMRI data for network node identification. Neuroimage 82, 403–415 (2013)

    Article  Google Scholar 

  20. Shen, X., et al.: Using connectome-based predictive modeling to predict individual behavior from brain connectivity. Nat. Protoc. 12(3), 506–518 (2017)

    Article  Google Scholar 

  21. Specht, K.: Current challenges in translational and clinical fMRI and future directions. Front. Psychiatry 10, 924 (2019)

    Article  Google Scholar 

  22. Steinhardt, K., et al.: Certified defenses for data poisoning attacks. In: Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  23. Tolpegin, V., Truex, S., Gursoy, M.E., Liu, L.: Data poisoning attacks against federated learning systems. In: Chen, L., Li, N., Liang, K., Schneider, S. (eds.) ESORICS 2020. LNCS, vol. 12308, pp. 480–501. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58951-6_24

    Chapter  Google Scholar 

  24. Wang, B., et al.: Neural cleanse: identifying and mitigating backdoor attacks in neural networks. In: IEEE Symposium on Security and Privacy, pp. 707–723 (2019)

    Google Scholar 

  25. Weis, S., Patil, K.R., Hoffstaedter, F., Nostro, A., Yeo, B.T.T., Eickhoff, S.B.: Sex classification by resting state brain connectivity. Cereb. Cortex 30(2), 824–835 (2020)

    Article  Google Scholar 

  26. Wynia, M.K., Cummins, D.S., VanGeest, J.B., Wilson, I.B.: Physician manipulation of reimbursement rules for patients: between a rock and a hard place. JAMA 283(14), 1858–1865 (2000)

    Article  Google Scholar 

  27. Zech, J.R., Badgeley, M.A., Liu, M., Costa, A.B., Titano, J.J., Oermann, E.K.: Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med. 15(11), e1002683 (2018)

    Article  Google Scholar 

  28. Zhang, Y., Liang, P.: Defending against whitebox adversarial attacks via randomized discretization. In: Chaudhuri, K., Sugiyama, M. (eds.) Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 89, pp. 684–693. PMLR (2019)

    Google Scholar 

Download references

Acknowledgements

This study was supported by R01MH121095 and the Wellcome Leap The First 1000 Days. Data used in the preparation of this article were obtained from the Adolescent Brain Cognitive Development (ABCD) Study (https://abcdstudy.org), held in the NIMH Data Archive (NDA). This is a multisite, longitudinal study designed to recruit more than 10,000 children age 9–10 and follow them over 10 years into early adulthood. The ABCD Study is supported by the National Institutes of Health and additional federal partners under award numbers U01DA041022, U01DA041028, U01DA041048, U01DA041089, U01DA041106, U01DA041117, U01DA041120, U01DA041134, U01DA041148, U01DA041156, U01DA041174, U24DA041123, and U24DA041147. A full list of supporters is available at a https://bcdstudy.org/nih-collaborators. A listing of participating sites and a complete listing of the study investigators can be found at https://abcdstudy.org/principal-investigators.html. ABCD consortium investigators designed and implemented the study and/or provided data but did not necessarily participate in analysis or writing of this report. This manuscript reflects the views of the authors and may not reflect the opinions or views of the NIH or ABCD consortium investigators. The ABCD data repository grows and changes over time. The ABCD data used in this report came from NIMH Data Archive Digital Object Identifier 10.15154/1504041. DOIs can be found at https://nda.nih.gov/study.html?id=721.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Yale University, New Haven, USA

    Matthew Rosenblatt & Dustin Scheinost

  2. Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, USA

    Dustin Scheinost

Authors
  1. Matthew Rosenblatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dustin Scheinost
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew Rosenblatt .

Editor information

Editors and Affiliations

  1. Université de Rennes 1, Rennes, France

    John S. H. Baxter

  2. Imperial College London, London, UK

    Islem Rekik

  3. Western University, London, ON, Canada

    Roy Eagleson

  4. University of Sydney, Sydney, NSW, Australia

    Luping Zhou

  5. IBM Almaden Research Center, San Jose, CA, USA

    Tanveer Syeda-Mahmood

  6. IBM Almaden Research Center, San Jose, CA, USA

    Hongzhi Wang

  7. University of San Francisco, San Francisco, CA, USA

    Mustafa Hajij

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rosenblatt, M., Scheinost, D. (2022). Data Poisoning Attack and Defenses in Connectome-Based Predictive Models. In: Baxter, J.S.H., et al. Ethical and Philosophical Issues in Medical Imaging, Multimodal Learning and Fusion Across Scales for Clinical Decision Support, and Topological Data Analysis for Biomedical Imaging. EPIMI ML-CDS TDA4BiomedicalImaging 2022 2022 2022. Lecture Notes in Computer Science, vol 13755. Springer, Cham. https://doi.org/10.1007/978-3-031-23223-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23223-7_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23222-0

  • Online ISBN: 978-3-031-23223-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation