Abstract
Diagnosis through imaging generally requires the combination of several modalities. Algorithms for data fusion allow merging information from different sources, mostly combining all images in a single step. In contrast, much less attention has been given to the incremental addition of new data descriptors, and the consideration of their costs (which can cover economic costs but also patient comfort and safety).
In this work, we formalise clinical diagnosis of a patient as a sequential process of decisions, each of these decisions being whether to take an additional acquisition, or, if there is enough information, to end the examination and produce a diagnosis. We formulate the goodness of a diagnosis process as a combination of the classification accuracy minus the cost of the acquired modalities. To obtain a policy, we apply reinforcement learning, which recommends the next modality to incorporate based on data acquired at previous stages and aiming at maximising the accuracy/cost trade-off. This policy therefore performs medical diagnosis and patient-wise feature selection simultaneously.
We demonstrate the relevance of this strategy on two binary classification datasets: a subset of a public heart disease database, including 531 instances with 11 scalar features, and a private echocardiographic dataset including signals from 5 standard image sequences used to assess cardiac function (2 speckle tracking, 2 flow Doppler and tissue Doppler), from 188 patients suffering hypertension, and 60 controls.
For each individual, our algorithm allows acquiring only the modalities relevant for the diagnosis, avoiding low-information acquisitions, which both resulted in higher stability of the chosen modalities and better classification performance under a limited budget.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bernard, O., et al.: Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans. Med. Imaging 37(11), 2514–2525 (2018)
Bernardino, G., et al.: Hierarchical multi-modality prediction model to assess obesity-related remodelling. In: Puyol Antón, E., Pop, M., Martín-Isla, C., Sermesant, M., Suinesiaputra, A., Camara, O., Lekadir, K., Young, A. (eds.) STACOM 2021. LNCS, vol. 13131, pp. 103–112. Springer, Cham (2022)
Detrano, R., et al.: International application of a new probability algorithm for the diagnosis of coronary artery disease. Am. J. Cardiol. 64(5), 304–310 (1989)
Garbi, M., et al.: EACVI appropriateness criteria for the use of cardiovascular imaging in heart failure derived from European National Imaging Societies voting. Europ. Heart J. Cardiovascular Imaging 17(7), 711–721 (2016)
Gong, W., et al.: Icebreaker: Element-wise Efficient Information Acquisition with a Bayesian Deep Latent Gaussian Model. In: Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc. (2019)
Hadian, M., Jabbari, A., Mazaheri, E., Norouzi, M.: What is the impact of clinical guidelines on imaging costs? J. Educ. Health Promotion 10, 10 (2021)
Loncaric, F., et al.: Automated pattern recognition in whole-cardiac cycle echocardiographic data: capturing functional phenotypes with machine learning. J. Am. Soc. Echocardiography 34(11), 1170–1183 (2021)
Nagueh, S.F., et al.: Recommendations for the evaluation of left ventricular diastolic function by echocardiography: an update from the American society of echocardiography and the European Association of Cardiovascular Imaging. J. Am. Soc. Echocardiography : official publication of the American Society of Echocardiography 29(4), 277–314 (2016)
Ng, A.Y.: Feature selection, L1 vs. L2 regularization, and rotational invariance. In: Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004 (2004)
Wang, J., Trapeznikov, K., Saligrama, V.: Efficient learning by directed acyclic graph for resource constrained prediction. In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28. Curran Associates, Inc. (2015)
Xu, Z.E., Kusner, M.J., Weinberger, K.Q., Chen, M., Chapelle, O.: Classifier cascades and trees for minimizing feature evaluation cost. J. Mach. Learn. Res. 15, 2113–2144 (2014)
Yin, H., Li, Y., Pan, S.J., Zhang, C., Tschiatschek, S.: Reinforcement Learning with Efficient Active Feature Acquisition. arXiv, September 2020
Zhou, T., Ruan, S., Canu, S.: A review: deep learning for medical image segmentation using multi-modality fusion. Array 3–4, 100004 (2019)
Acknowledgements
The authors acknowledge the partial support from the French ANR (LABEX PRIMES of Univ. Lyon [ANR-11-LABX-0063] and the JCJC project “MIC-MAC” [ANR-19-CE45-0005]) and the Spanish AEI [PID2019-108141GB-I00]. We thank Prof. B. Bijnens (IDIBAPS & ICREA, Barcelona, Spain) for fruitful discussions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bernardino, G. et al. (2022). Reinforcement Learning for Active Modality Selection During Diagnosis. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13431. Springer, Cham. https://doi.org/10.1007/978-3-031-16431-6_56
Download citation
DOI: https://doi.org/10.1007/978-3-031-16431-6_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16430-9
Online ISBN: 978-3-031-16431-6
eBook Packages: Computer ScienceComputer Science (R0)