Abstract
The use of lung ultrasound imaging has recently emerged as a quick, cost-effective, and safe method for diagnosis of patients with COVID-19. Challenges with training deep networks to identify COVID-19 signatures in lung ultrasound data are that large datasets do not yet exist; disease signatures are sparse, but are spatially and temporally correlated; and signatures may appear sporadically in ultrasound video sequences. We propose an attention-based video model that is specifically designed to detect these disease signatures, and leverage a knowledge transfer approach to overcome existing limitations in data availability. In our design, a convolutional neural network extracts spatially encoded features, which are fed to a transformer encoder to capture temporal information across the frames and focus on the most important frames. We guide the network to learn clinically relevant features by training it on a pulmonary biomarker detection task, and then transferring the model’s knowledge learned from this problem to achieve 80% precision and 87% recall for COVID-19. Our results outperform the state-of-the-art model on a public lung ultrasound dataset. We perform ablation studies to highlight the efficacy of our design over previous state-of-the-art frame-based approaches. To demonstrate that our approach learns clinically relevant imaging biomarkers, we introduce a novel method for generating attention-based video classification explanations called Biomarker Attention-scaled Class Activation Mapping (Bio-AttCAM). Our analysis of the activation map shows high correlation with the key frames selected by clinicians.
T. Lum and M. Mahdavi contributed equally to this work.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abrams, E.R., Rose, G., Fields, J.M., Esener, D.: Point-of-care ultrasound in the evaluation of COVID-19. J. Emerg. Med. 2020, 403–408 (2020)
Amann, J., Blasimme, A., Vayena, E., Frey, D., Madai, V.: Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med. Inform. Decision Making 20, 310 (2020)
Bach, S., et al.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, 1–46 (2015)
Born, J., et al.: Accelerating detection of lung pathologies with explainable ultrasound image analysis. Appl. Sci. 11(2), 672 (2021)
Brattain, L.J., Telfer, B.A., Dhyani, M., Grajo, J.R., Samir, A.E.: Machine learning for medical ultrasound: status, methods, and future opportunities. Abdom Radiol. (NY) 2018, 786–799 (2018)
Buda, N., Segura-Grau, E., Cylwik, J., Wełnickid, M.: Lung ultrasound in the diagnosis of COVID-19 infection - a case series and review of the literature. Adv. Med. Sci. 2020, 378–385 (2020)
Dong, E., Du, H., Gardner, L.: An interactive web-based dashboard to track Covid-19 in real time. Lancet Infect. Dis. 20(5), 533–534 (2020)
Hiley, L., Preece, A., Hicks, Y.: Explainable deep learning for video recognition tasks: a framework & recommendations. arXiv preprint arXiv:1909.05667 (2019)
Li, J., Qiu, H.: Comparing attention-based neural architectures for video captioning (2019)
Liu, T., Shao, Q.: BERT for large-scale video segment classification with test-time augmentation. arXiv preprint arXiv:1912.01127 (2019)
Peng, Q., et al.: Findings of lung ultrasonography of novel corona virus pneumonia during the 2019–2020 epidemic. Intensive Care Med. 46, 849–850 (2020)
Roy, S., et al.: Deep learning for classification and localization of COVID-19 markers in point-of-care lung ultrasound. IEEE Trans. Med. Imag. 39(8), 2676–2687 (2020)
Selvaraju, R.R., et al.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 618–626 (2017)
Acknowledgement
This work was supported by Canada’s Digital Technology Supercluster, the Natural Sciences and Engineering Research Council of Canada (NSERC), and the Canadian Institutes of Health Research (CIHR).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lum, T. et al. (2021). Imaging Biomarker Knowledge Transfer for Attention-Based Diagnosis of COVID-19 in Lung Ultrasound Videos. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, SL., Hu, Y. (eds) Simplifying Medical Ultrasound. ASMUS 2021. Lecture Notes in Computer Science(), vol 12967. Springer, Cham. https://doi.org/10.1007/978-3-030-87583-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-87583-1_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87582-4
Online ISBN: 978-3-030-87583-1
eBook Packages: Computer ScienceComputer Science (R0)