Abstract
Mainstream segmentation algorithms currently rely on supervised learning and thus require large pixel-labelled datasets for training. However, manually labelling regions of interest in medical images is both time-consuming and expertise-demanding, compressing the scale of the dataset and thus limiting the accuracy of medical image segmentation. Self-supervised learning is often preferred in cases where annotation dependency needs to be alleviated or budget is limited. However, existing methods have ignored the properties of medical images and lack adaptive masking methods, resulting in poor generalisation. This paper proposes a self-supervised pre-training framework for retinal vessel segmentation based on a priori knowledge. The proposed framework is called KnowMIM and works in two phases guided with knowledge: (1) Adaptive masks generation. KnowMIM utilises an edge detection algorithm to extract the location of vessel contours as a priori information, and then generates an adaptive mask for each retinal image, which is masked for data augmentation, and (2) Masked image modeling. KnowMIM carries out masked image modeling via a U-Net architecture and performs reconstruction through self-supervised learning to pre-train the encoder and decoder. Extensive experiments have been conducted on public retinal datasets for vessels segmentation. Results demonstrate that KnowMIM outperforms state-of-the-art pre-training counterparts. Additionally, KnowMIM effectively utilises unlabelled data and exhibits generalisation on external datasets.
J. Zhu and W. Chen—Contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aggarwal, V., Gupta, A.: Integrating morphological edge detection and mutual information for nonrigid registration of medical images. Curr. Med. Imaging 15(3), 292–300 (2019)
Asano, Y.M., Rupprecht, C., Vedaldi, A.: Self-labelling via simultaneous clustering and representation learning. arXiv preprint arXiv:1911.05371 (2019)
Azizi, S., et al.: Big self-supervised models advance medical image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3478–3488 (2021)
Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 132–149 (2018)
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. In: Advances in Neural Information Processing Systems, vol. 33, pp. 9912–9924 (2020)
Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., Rueckert, D.: Self-supervised learning for medical image analysis using image context restoration. Med. Image Anal. 58, 101539 (2019)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)
Chen, T., Kornblith, S., Swersky, K., Norouzi, M., Hinton, G.E.: Big self-supervised models are strong semi-supervised learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 22243–22255 (2020)
Chen, W., Li, C., Chen, D., Luo, X.: A knowledge-based learning framework for self-supervised pre-training towards enhanced recognition of biomedical microscopy images. Neural Netw. 167, 810–826 (2023)
Chen, X., Yao, L., Zhou, T., Dong, J., Zhang, Y.: Momentum contrastive learning for few-shot covid-19 diagnosis from chest CT images. Pattern Recogn. 113, 107826 (2021)
Chen, X., Fan, H., Girshick, R., He, K.: Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 (2020)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Grill, J.B., et al.: Bootstrap your own latent-a new approach to self-supervised learning. In: Advances in Neural Information Processing Systems, vol. 33, pp. 21271–21284 (2020)
Guo, C., Szemenyei, M., Yi, Y., Wang, W., Chen, B., Fan, C.: SA-UNet: spatial attention u-net for retinal vessel segmentation. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 1236–1242. IEEE (2021)
Haghighi, F., Taher, M.R.H., Gotway, M.B., Liang, J.: DiRA: discriminative, restorative, and adversarial learning for self-supervised medical image analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20824–20834 (2022)
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Jing, L., Tian, Y.: Self-supervised visual feature learning with deep neural networks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(11), 4037–4058 (2020)
Kipli, K., et al.: A review on the extraction of quantitative retinal microvascular image feature. Comput. Math. Methods Med. 2018 (2018)
Liu, X., et al.: Self-supervised learning: generative or contrastive. IEEE Trans. Knowl. Data Eng. 35(1), 857–876 (2021)
Mao, J., Guo, S., Chang, Y., Yin, X., Nie, B.: Medical supervised masked autoencoders: crafting a better masking strategy and efficient fine-tuning schedule for medical image classification. arXiv preprint arXiv:2305.05871 (2023)
Misra, I., Maaten, L.V.D.: Self-supervised learning of pretext-invariant representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6707–6717 (2020)
Noroozi, M., Vinjimoor, A., Favaro, P., Pirsiavash, H.: Boosting self-supervised learning via knowledge transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9359–9367 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Tian, Y., Krishnan, D., Isola, P.: Contrastive multiview coding. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12356, pp. 776–794. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_45
Yu, H., Shim, J.H., Kwak, J., Song, J.W., Kang, S.J.: Vision transformer-based retina vessel segmentation with deep adaptive gamma correction. In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1456–1460. IEEE (2022)
Yuan, M., et al.: Devil is in the queries: advancing mask transformers for real-world medical image segmentation and out-of-distribution localization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23879–23889 (2023)
Zbontar, J., Jing, L., Misra, I., LeCun, Y., Deny, S.: Barlow twins: self-supervised learning via redundancy reduction. In: International Conference on Machine Learning, pp. 12310–12320. PMLR (2021)
Zhang, C., Zhang, C., Song, J., Yi, J.S.K., Kweon, I.S.: A survey on masked autoencoder for visual self-supervised learning. In: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), pp. 6805–6813 (2023)
Zhou, H.Y., Lu, C., Yang, S., Han, X., Yu, Y.: Preservational learning improves self-supervised medical image models by reconstructing diverse contexts. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3499–3509 (2021)
Zhou, Z., et al.: Models genesis: generic autodidactic models for 3D medical image analysis. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 384–393. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_42
Acknowledgement
This research was funded by the Natural Science Foundation of Hunan Province of China (No. 2022JJ30666).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, J. et al. (2024). KnowMIM: a Self-supervised Pre-training Framework Based on Knowledge-Guided Masked Image Modeling for Retinal Vessel Segmentation. In: Wand, M., Malinovská, K., Schmidhuber, J., Tetko, I.V. (eds) Artificial Neural Networks and Machine Learning – ICANN 2024. ICANN 2024. Lecture Notes in Computer Science, vol 15023. Springer, Cham. https://doi.org/10.1007/978-3-031-72353-7_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-72353-7_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72352-0
Online ISBN: 978-3-031-72353-7
eBook Packages: Computer ScienceComputer Science (R0)