Abstract
We explore the potential of deep convolutional neural network (CNN) models for differential diagnosis of gout from musculoskeletal ultrasound (MSKUS), as no prior study on this topic is known. Our exhaustive study of state-of-the-art (SOTA) CNN image classification models for this problem reveals that they often fail to learn the gouty MSKUS features, including the double contour sign, tophus, and snowstorm, which are essential for sonographers’ decisions. To address this issue, we establish a framework to adjust CNNs to “think like sonographers” for gout diagnosis, which consists of three novel components: (1) Where to adjust: Modeling sonographers’ gaze map to emphasize the region that needs adjust; (2) What to adjust: Classifying instances to systematically detect predictions made based on unreasonable/biased reasoning and adjust; (3) How to adjust: Developing a training mechanism to balance gout prediction accuracy and attention reasonability for improved CNNs. The experimental results on clinical MSKUS datasets demonstrate the superiority of our method over several SOTA CNNs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baumgartner, C.F., et al.: Sononet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound. IEEE Trans. Med. Imaging 36(11), 2204–2215 (2017)
Bylinskii, Z., Judd, T., Oliva, A., Torralba, A., Durand, F.: What do different evaluation metrics tell us about saliency models? IEEE Trans. Pattern Anal. Mach. Intell. 41(3), 740–757 (2018)
Cai, Y., Sharma, H., Chatelain, P., Noble, J.A.: Multi-task SonoEyeNet: detection of fetal standardized planes assisted by generated sonographer attention maps. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 871–879. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_98
Cai, Y., Sharma, H., Chatelain, P., Noble, J.A.: Sonoeyenet: standardized fetal ultrasound plane detection informed by eye tracking. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 1475–1478. IEEE (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Liu, S., et al.: Deep learning in medical ultrasound analysis: a review. Engineering 5(2), 261–275 (2019)
Lou, J., Lin, H., Marshall, D., Saupe, D., Liu, H.: Transalnet: towards perceptually relevant visual saliency prediction. Neurocomputing 494, 455–467 (2022)
Mall, S., Brennan, P.C., Mello-Thoms, C.: Modeling visual search behavior of breast radiologists using a deep convolution neural network. J. Med. Imaging 5(3), 035502–035502 (2018)
Mall, S., Krupinski, E., Mello-Thoms, C.: Missed cancer and visual search of mammograms: what feature-based machine-learning can tell us that deep-convolution learning cannot. In: Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment, vol. 10952, pp. 281–287. SPIE (2019)
Patra, A., et al.: Efficient ultrasound image analysis models with sonographer gaze assisted distillation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 394–402. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_43
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, S., Ouyang, X., Liu, T., Wang, Q., Shen, D.: Follow my eye: using gaze to supervise computer-aided diagnosis. IEEE Trans. Med. Imaging 41(7), 1688–1698 (2022)
Acknowledgment
This work was supported in part by National Nature Science Foundation of China grants (62271246, U20A20389, 82027807, U22A2051), Key Research and Development Plan of Jiangsu Province (No. BE2022842), National Key Research and Development Program of China (2022YFC2405200).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, Z. et al. (2023). Thinking Like Sonographers: A Deep CNN Model for Diagnosing Gout from Musculoskeletal Ultrasound. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14225. Springer, Cham. https://doi.org/10.1007/978-3-031-43987-2_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-43987-2_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43986-5
Online ISBN: 978-3-031-43987-2
eBook Packages: Computer ScienceComputer Science (R0)
