Abstract
The objects in fluorescence microscopy images (FMIs) are rather densely populated, and often have complex structures as well as large color variations. These issues result in time-consuming manual annotations for training a segmentation model. Few-shot segmentation (FSS) models reduce the requirements of annotations by using prototypical networks to generalize the object regions given only a few annotation masks. These methods, however, often fail to generalize to FMIs due to the domain shift from the images of source class to those of target class. To tackle these challenges, we adapt the prototypical networks from the few-shot learning scheme as our base model architecture to learn a compact feature representation of the object, with only a few annotated masks. Then, to solve the domain shift problem, we propose a resampling strategy to regularize the model training. Specifically, we construct the training set using not only the images form the source class, but also the support set from the target class by resampling the latter for regularizing the training. To distinguish our FMIs segmentation method from the conventional FSS models, we define it as a training regularization strategy based on the prototypical networks (TRPN). Experimental results on three public datasets, i.e.the ER, MITO and NUC, have confirmed the advantages of our method, which provides robust and accurate segmentation performance on FMIs only with a few annotation masks. Source code and data of this work are made publicly available at: https://github.com/cbmi-group/TRPN-model.
This work was supported in part by research grants from Natural Science Foundation of China (No. 32101216, No. 91954201, No. 31971289), the Chinese Academy of Sciences (No. 292019000056), University of Chinese Academy of Sciences (No. 115200M001), and the Beijing Municipal Science & Technology Commission (No. 5202022).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The MITO can present diverse shapes, like ribbon, pole, reticulation, etc..
- 2.
Here, we use testing and query to indicate the identical definition that represents the image to be segmented.
References
Moen, E., Bannon, D., Kudo, T., et al.: Deep learning for cellular image analysis. Nat. Methods 16(12), 1233–1246 (2019)
Nixon-Abell, j., Obara, C.J., Weigel, A.V., et al.: Increased spatiotemporal resolution reveals highly dynamic dense tubular matrices in the peripheral ER. Science 354(6311), aaf3928 (2016)
Caicedo, J.C., Goodman, A., Karhohs, K.W., et al.: Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nat. Methods 16(12), 1247–1253 (2019)
Greenwald, N.F., Miller, G., Moen, E., et al.: Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat. Biotechnol. 40(4), 555–565 (2022)
Wang, Y., Yao, Q., Kwok, J.T., Ni, L.M.: Generalizing from a few examples: a survey on few-shot learning. ACM Comput. Surv. 53(3), 1–34 (2020)
Cheng, G., Li, R., Lang, C., Han, J.: Task-wise attention guided part complementary learning for few-shot image classification. Sci. China Inf. Sci. 64(2), 1–14 (2021). https://doi.org/10.1007/s11432-020-3156-7
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Dong , N., Xing, E.P.: Few-shot semantic segmentation with prototype learnings. In: British Machine Vision Conference, vol. 3 (2018)
Nguyen, K., Todorovic, S.: Feature weighting and boosting for few-shot segmentation. In: IEEE International Conference on Computer Vision, pp. 622–631 (2019)
Tian, Z., Zhao, H., Shu, M., et al.: Prior guided feature enrichment network for few-shot segmentation. IEEE Trans. Pattern. Anal. Mach. Intell. (2020)
Hospedales, T., Antoniou, A., Micaelli, P., Storkey, A.: Meta-learning in neural networks: a survey. arXiv preprint arXiv:2004.05439 (2020)
Weiss, K., Khoshgoftaar, T.M., Wang, D.D.: A survey of transfer learning. J. Big Data 3(1), 1–40 (2016). https://doi.org/10.1186/s40537-016-0043-6
Zhang, X., Wei, Y., Yang, Y., Huang, T.S.: SG-One: similarity guidance network for one-shot semantic segmentation. IEEE Trans. Cybern. 50(9), 3855–3865 (2020)
Ouyang, C., Biffi, C., Chen, C., et al.: Self-supervised learning for few-shot medical image segmentation. IEEE Trans. Med. Imaging 41(7), 1837–1848 (2022)
Guo, Y., Huang, J., Zhou, Y., Luo, Y., Li, W., Yang, G.: Segmentation of intracellular structures in fluorescence microscopy images by fusing low-level features. In: Chinese Conference on Pattern Recognition and Computer Vision, vol. 13021, pp. 386–397. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88010-1_32
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR (2017)
Jamal, M.A., Qi, G.J.: Task agnostic meta-learning for few-shot learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 11719–11727 (2019)
Sun, Q., Liu, Y., Chua, T., Schiele, B.: Meta-transfer learning for few-shot learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 403–412 (2019)
Siam, M., Oreshkin, B.N., Jagersand, M.: AMP: adaptive masked proxies for few-shot segmentation. In: IEEE International Conference on Computer Vision, pp. 5249–5258 (2019)
Zhang, C., Lin, G., Liu, F., et al.: Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation. In: IEEE International Conference on Computer Vision, pp. 9587–9595 (2019)
Liu, W., Zhang, C., Lin, G., Liu, F.: CRNet: cross-reference networks for few-shot segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4165–4173 (2020)
Zhang, B., Xiao, J., Qin, T.: Self-guided and cross-guided learning for few-shot segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8312–8321 (2021)
Liu, Y., Zhang, X., Zhang, S., He, X.: Part-Aware prototype network for few-shot semantic segmentation. In: Europe Conference on Computer Vision, vol. 12354, pp. 142–158. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_9
Yang, B., Liu, C., Li, B., Jiao, J., Ye, Q.: Prototype mixture models for few-shot semantic segmentation. In: Europe Conference on Computer Vision, vol. 12353, pp. 763–778. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58598-3_45
Yang, Y., Meng, F., Li, H., Wu, Q., Xu, X., Chen, S.: A new local transformation module for few-shot segmentation. In: International Conference on Multimedia Modeling, vol. 11962, pp. 76–87. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37734-2_7
Lang, C., Cheng, G., Tu, B., Han, J.: Learning what not to segment: a new perspective on few-shot segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8057–8067 (2022)
Shen, Z., Liu, Z., Qin, J., et al.: Partial is better than all: revisiting fine-tuning strategy for few-shot learning. In: AAAI Conference on Artificial Intelligence, vol. 35, pp. 9594–9602 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-Net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1
Wang, J., Sun, K., Cheng, T., et al.: Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern. Anal. Mach. Intell. 43(10), 3349–3364 (2020)
Paszke, A., Gross, S., Massa, F., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Wang, Z., Xu, J., Liu, L., et al.: RANet: ranking attention network for fast video object segmentation. In: IEEE International Conference on Computer Vision, pp. 3978–3987 (2019)
Arslan, S., Ersahin, T., Cetin-Atalay, R., Gunduz-Demir, C.: Attributed relational graphs for cell nucleus segmentation in fluorescence microscopy images. IEEE Trans. Med. Imaging 32(6), 1121–1131 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, Y., Luo, Y., Li, W., Yang, G. (2022). Fluorescence Microscopy Images Segmentation Based on Prototypical Networks with a Few Annotations. In: Yu, S., et al. Pattern Recognition and Computer Vision. PRCV 2022. Lecture Notes in Computer Science, vol 13535. Springer, Cham. https://doi.org/10.1007/978-3-031-18910-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-18910-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18909-8
Online ISBN: 978-3-031-18910-4
eBook Packages: Computer ScienceComputer Science (R0)