Abstract
Deep learning automation of medical image analysis is highly desirable for purposes including organ/tissue segmentation and disease detection. However, deep learning traditionally relies on supervised training methods, while medical images are far more expensive to label than natural images. Self-supervised learning (SSL) has been gaining attention as a technique that allows strong model performance with only a small amount of labeled data. This would be particularly useful in ultrasound (US) imaging, which can involve hundreds of images per video sweep, saving time and money for labeling.
In this paper, we proposed a new SSL-based image segmentation technique that can be applied to bone segmentation in wrist US. This is the first use of the classification models SSL pretraining method SimMIM in wrist US. We modified the SimMIM SSL pretraining architecture, used a speckle noise masking policy to generate noise artifacts similar to those seen in US, changed the loss function, and analyzed how they influenced the downstream segmentation tasks.
Using modified SimMIM, our approach surpassed the performance of state-of-the-art fully supervised models on wrist bony region segmentation by up to 3.2% higher Dice score and up to 4.5% higher Jaccard index, using an extremely small labeled dataset with only 187/935 images and generated labels visually consistent with human labeling on the test set of 3822 images. The SSL pretrained models were also robust on the test set annotated by different medical experts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Almalki, A., Latecki, L.J.: Self-supervised learning with masked image modeling for teeth numbering, detection of dental restorations, and instance segmentation in dental panoramic radiographs. In: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 5583–5592. IEEE (2023)
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, June 2020. https://doi.org/10.48550/arXiv.2006.09882
Champagne, N., Eadie, L., Regan, L., Wilson, P.: The effectiveness of ultrasound in the detection of fractures in adults with suspected upper or lower limb injury: a systematic review and subgroup meta-analysis. BMC Emergency Med. 19(1), 17 (2019)
Chen, J., et al.: TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation, February 2021. https://doi.org/10.48550/arXiv.2102.04306, arXiv:2102.04306 [cs]
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A Simple Framework for Contrastive Learning of Visual Representations, February 2020. https://doi.org/10.48550/arXiv.2002.05709
Chen, X., He, K.: Exploring Simple Siamese Representation Learning, November 2020. https://doi.org/10.48550/arXiv.2011.10566
Dosovitskiy, A., et al.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, October 2020. https://doi.org/10.48550/arXiv.2010.11929
El-Nouby, A., Izacard, G., Touvron, H., Laptev, I., Jegou, H., Grave, E.: Are Large-scale Datasets Necessary for Self-Supervised Pre-training? https://doi.org/10.48550/arXiv.2112.10740
Felfeliyan, B., et al.: Self-Supervised-RCNN for Medical Image Segmentation with Limited Data Annotation, July 2022. https://doi.org/10.48550/arXiv.2207.11191
Gebhardt, C., et al.: Femur reconstruction in 3D ultrasound for orthopedic surgery planning, 18(6), 1001–1008. https://doi.org/10.1007/s11548-023-02868-4
Goodfellow, I.J., et al.: Generative Adversarial Networks, June 2014. https://doi.org/10.48550/arXiv.1406.2661
Grill, J.B., et al. : Bootstrap your own latent: a new approach to self-supervised Learning, June 2020. https://doi.org/10.48550/arXiv.2006.07733
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked Autoencoders Are Scalable Vision Learners, November 2021. https://doi.org/10.48550/arXiv.2111.06377
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9726–9735. IEEE, Seattle, WA, USA (2020)
Hedström, E.M., Svensson, O., Bergström, U., Michno, P.: Epidemiology of fractures in children and adolescents: increased incidence over the past decade: a population-based study from northern Sweden. Acta Orthop. 81(1), 148–153 (2010)
Jaremko, J.L., Hareendranathan, A., Bolouri, S.E.S., Frey, R.F., Dulai, S., Bailey, A.L.: AI aided workflow for hip dysplasia screening using ultrasound in primary care clinics, 13(1), 9224. https://doi.org/10.1038/s41598-023-35603-9
Knight, J., et al.: 2D/3D ultrasound diagnosis of pediatric distal radius fractures by human readers vs artificial intelligence. Sci. Rep. 13, 14535 (2023). https://doi.org/10.1038/s41598-023-41807-w
LeCun, Y., et al.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems, vol. 2. Morgan-Kaufmann (1989)
Liu, X., et al.: Self-supervised learning: generative or contrastive, 35(1), 857–876 (2021)
Malhotra, P., Gupta, S., Koundal, D., Zaguia, A., Enbeyle, W.: Deep neural networks for medical image segmentation. J. Healthc. Eng. 2022, 9580991 (2022)
Meena, S., Sharma, P., Sambharia, A.K., Dawar, A.: Fractures of distal radius: an overview. J. Family Med. Primary Care 3(4), 325 (2014)
Ouyang, C., Biffi, C., Chen, C., Kart, T., Qiu, H., Rueckert, D.: Self-supervised learning for few-shot medical image segmentation, 41(7), 1837–1848 (2022)
Pandey, P.U., Quader, N., Guy, P., Garbi, R., Hodgson, A.J.: Ultrasound bone segmentation: a scoping review of techniques and validation practices, 46(4), 921–935 (2020)
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2536–2544. IEEE, Las Vegas, NV, USA, June 2016
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation, May 2015. arXiv:1505.04597 [cs]
Tang, Y., et al.: Self-supervised pre-training of swin transformers for 3D medical image analysis, pp. 20730–20740 (2022)
du Toit, C., Orlando, N., Papernick, S., Dima, R., Gyacskov, I., Fenster, A.: Automatic femoral articular cartilage segmentation using deep learning in three-dimensional ultrasound images of the knee, 4(3), 100290. https://doi.org/10.1016/j.ocarto.2022.100290
Wei, C., Fan, H., Xie, S., Wu, C.Y., Yuille, A., Feichtenhofer, C.: Masked Feature Prediction for Self-Supervised Visual Pre-Training. pp. 14668–14678
Xie, Z., et al.: SimMIM: A Simple Framework for Masked Image Modeling, November 2021. https://doi.org/10.48550/arXiv.2111.09886
Zhang, J., Boora, N., Melendez, S., Rakkunedeth Hareendranathan, A., Jaremko, J.: Diagnostic accuracy of 3D ultrasound and artificial intelligence for detection of pediatric wrist injuries. Children (Basel, Switzerland) 8(6), 431 (2021)
Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 649–666. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_40
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, Y. et al. (2023). Self-Supervised Learning to More Efficiently Generate Segmentation Masks for Wrist Ultrasound. In: Kainz, B., Noble, A., Schnabel, J., Khanal, B., Müller, J.P., Day, T. (eds) Simplifying Medical Ultrasound. ASMUS 2023. Lecture Notes in Computer Science, vol 14337. Springer, Cham. https://doi.org/10.1007/978-3-031-44521-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-44521-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44520-0
Online ISBN: 978-3-031-44521-7
eBook Packages: Computer ScienceComputer Science (R0)
