Abstract
Detection and segmentation of cells can be of great significance to the further quantitative analysis of biomedical research in the field of biomedical engineering. Especially, it is a serious challenging task for some microscope imaging devices with limited resources owing to a large number of learning parameters and computational burden when using the detect-then-segment two-stage strategy. In this work, an anchor-free instance segmentation method is proposed for cells based on mask contour. Specifically, an anchor-free network framework is firstly designed for instance segmentation by replacing the standard convolution in the contour generation branch with a deformable convolution. Then, these key contour points are linked to generate coarse contours of cells by using a Graham algorithm. Thirdly, the obtained coarse contours are used for regressing and approaching the ground truths in polar coordinates under the supervision of the Mask loss function. Finally, a series of comparison experiments are conducted to verify the effectiveness of the proposed methods on various datasets. The results show that the proposed method can obtain a better trade-off between recognition performance and computing efficiency, and it can surpass the existing one-stage SOTA methods in the Dice coefficient while maintaining higher computational efficiency on different datasets. Even with a two-stage instance segmentation method like Mask R-CNN, the proposed method can only obtain slightly lower Dice coefficients but with much higher FPS.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability Statements
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
Hu H, Guan Q, Chen S, Ji Z, Lin Y (2020) Detection and recognition for life state of cell cancer using two-stage cascade cnns. IEEE/ACM Trans Comput Biol Bioinf 17(3):887–898
Upschulte E, Harmeling S, Amunts K, Dickscheid T (2022) Contour proposal networks for biomedical instance segmentation. Med Image Anal 102371
Prangemeier T, Reich C, Koeppl H (2020) Attention-based transformers for instance segmentation of cells in microstructures. In: 2020 IEEE international conference on bioinformatics and biomedicine (BIBM), pp 700–707
Zhao M, Jha A, Liu Q, Millis BA, Mahadevan-Jansen A, Lu L, Landman BA, Tyska MJ, Huo Y (2021) Faster mean-shift: Gpu-accelerated clustering for cosine embedding-based cell segmentation and tracking. Med Image Anal 71:102048
Xie E, Wang W, Ding M, Zhang R, Luo P (2021) Polarmask++: enhanced polar representation for single-shot instance segmentation and beyond. IEEE Trans Pattern Anal Mach Intell
Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T, Gao S, Liu J (2019) Ce-net: context encoder network for 2d medical image segmentation. IEEE Trans Med Imaging 38(10):2281–2292
Schmidt U, Weigert M, Broaddus C, Myers G (2018) Cell detection with star-convex polygons. In: International conference on medical image computing and computer-assisted intervention, pp 265–273
Lux F, Matula P (2020) Cell segmentation by combining marker-controlled watershed and deep learning. arXiv preprint arXiv:2004.01607
Chen L, Strauch M, Merhof D (2019) Instance segmentation of biomedical images with an object-aware embedding learned with local constraints. In: International conference on medical image computing and computer-assisted intervention, pp 451–459
Lee H, Jeong W-K (2020) Scribble2label: scribble-supervised cell segmentation via self-generating pseudo-labels with consistency. In: International conference on medical image computing and computer-assisted intervention, pp 14–23
Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD (2020) Doubleu-net: a deep convolutional neural network for medical image segmentation. In: 2020 IEEE 33rd international symposium on computer-based medical systems (CBMS), pp 558–564
Yang X, Yang Y, Ma S, Li Z, Dong W, Woźniak M (2024) Samt-generator: a second-attention for image captioning based on multi-stage transformer network. Neurocomputing 593:127823
Yu X, Ye X, Zhang S (2022) Floating pollutant image target extraction algorithm based on immune extremum region. Digit Signal Process 123:103442
Yu X, Lu Y, Gao Q (2021) Pipeline image diagnosis algorithm based on neural immune ensemble learning. Int J Pres Ves Pip 189:104249
Bai D, Xia D, Huang D, Hu Y, Li Y, Li H (2023) Spatial-temporal graph neural network based on gated convolution and topological attention for traffic flow prediction. Appl Intell 53(24):30843–30864
Wang R, Lei T, Cui R, Zhang B, Meng H, Nandi AK (2020) Medical image segmentation using deep learning: a survey. IET Image Process
Petukhov V, Xu RJ, Soldatov RA, Cadinu P, Khodosevich K, Moffitt JR, Kharchenko PV (2022) Cell segmentation in imaging-based spatial transcriptomics. Nat Biotechnol 40(3):345–354
Greenwald NF, Miller G, Moen E, Kong A, Kagel A, Dougherty T, Fullaway CC, McIntosh BJ, Leow KX, Schwartz MS et al (2022) Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat Biotechnol 40(4):555–565
Prangemeier T, Wildner C, Françani AO, Reich C, Koeppl H (2022) Yeast cell segmentation in microstructured environments with deep learning. Biosystems 211:104557
Zhang Y, Higashita R, Fu H, Xu Y, Zhang Y, Liu H, Zhang J, Liu J (2021) A multi-branch hybrid transformer network for corneal endothelial cell segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 99–108
Wang Z, Wang Z (2021) Robust cell segmentation based on gradient detection, gabor filtering and morphological erosion. Biomed Signal Process Control 65:102390
Xie E, Sun P, Song X, Wang W, Liu X, Liang D, Shen C, Luo P (2020) Polarmask: single shot instance segmentation with polar representation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12193–12202
Zhao M, Liu Q, Jha A, Deng R, Yao T, Mahadevan-Jansen A, Tyska MJ, Millis BA, Huo Y (2021) Voxelembed: 3d instance segmentation and tracking with voxel embedding based deep learning. In: International workshop on machine learning in medical imaging, pp 437–446
Hu H, Liu A, Zhou Q, Guan Q, Li X, Chen Q (2021) An adaptive learning method of anchor shape priors for biological cells detection and segmentation. Comput Methods Programs Biomed 208:106260
Yuling X, Ning W, Shaohua W, Xiaoming W, Wang P, Yanning Z (2023) Searching sharing relationship for instance segmentation decoder. Appl Intell 53:20938–20949
He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969
Zhou X, Zhuo J, Krahenbuhl P (2019) Bottom-up object detection by grouping extreme and center points. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 850–859
Liu S, Qi X, Shi J, Zhang H, Jia J (2016) Multi-scale patch aggregation (mpa) for simultaneous detection and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3141–3149
Bancher B, Mahbod A, Ellinger I, Ecker R, Dorffner G (2021) Improving mask r-cnn for nuclei instance segmentation in hematoxylin & eosin-stained histological images. In: MICCAI workshop on computational pathology, pp 20–35
Xie E, Wang W, Ding M, Zhang R, Luo P (2021) Polarmask++: enhanced polar representation for single-shot instance segmentation and beyond. IEEE Trans Pattern Anal Mach Intell 44(9):5385–5400
Dong X, Ouyang Z, Guo Z, Niu J (2021) Polarmask-tracker: lightweight multi-object tracking and segmentation model for edge device. In: 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom), pp 689–696
Li G-j, Zheng J-h, Zhang G-z (2010) Design and implementation of an improved algorithm of convex hull. Modern Comput
Ren S, He K, Girshick RB, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: NIPS, pp 91–99
Lee Y, Park J (2020) Centermask: real-time anchor-free instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13906–13915
Chen X, Girshick R, He K, Dollár P (2019) Tensormask: a foundation for dense object segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2061–2069
Bolya D, Zhou C, Xiao F, Lee YJ (2019) Yolact: real-time instance segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9157–9166
Jiang H, Zhang R, Zhou Y, Wang Y, Chen H (2023) Donet: deep de-overlapping network for cytology instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 15641–15650
Graham S, Vu QD, Raza SEA, Azam A, Tsang YW, Kwak JT, Rajpoot N (2019) Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med Image Anal 58:101563
Hörst F, Rempe M, Heine L, Seibold C, Keyl J, Baldini G, Ugurel S, Siveke J, Grünwald B, Egger J et al (2024) Cellvit: vision transformers for precise cell segmentation and classification. Med Image Anal 94:103143
Li Y, Qi H, Dai J, Ji X, Wei Y (2017) Fully convolutional instance-aware semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2359–2367
Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8759–8768
Huang Z, Huang L, Gong Y, Huang C, Wang X (2019) Mask scoring r-cnn. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6409–6418
Tian Z, Shen C, Chen H, He T (2019) Fcos: fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9627–9636
Zhan J, Luo Y, Guo C, Wu Y, Meng J, Liu J (2024) Yolopx: anchor-free multi-task learning network for panoptic driving perception. Pattern Recogn 148:110152
Linfeng S, Yan L, Xi Z (2023) Gsa-dla34: a novel anchor-free method for human-vehicle detection. Appl Intell 53:24619–24637
Wang K-L, Chou P-H, Chou Y-C, Liu C-J, Lin C-K, Tseng Y-C (2024) Mp-polarmask: a faster and finer instance segmentation for concave images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3705–3714
Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 764–773
Liu Y, Wang X, Zhu Y, Fu X, Zha Z-J (2024) Sdcnet: spatially-adaptive deformable convolution networks for hr nonhomogeneous dehazing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) workshops, pp 6682–6691
Zhuo S, Zhang J (2024) Attention-based deformable convolutional network for chinese various dynasties character recognition. Expert Syst Appl 238:121881
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125
Maška M et al (2014) A benchmark for comparison of cell tracking algorithms. Bioinformatics 30(11):1609–1617
Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision, pp 740–755
Paszke A, Gross S, Chintala S, Chanan G, Yang, E, DeVito (2017) Automatic differentiation in pytorch
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee, pp 248–255
Huang Z, Huang L, Gong Y, Huang C, Wang X (2019) Mask scoring r-cnn. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 6402–6411
Wang X, Zhang R, Kong T, Li L, Shen C (2020) Solov2: dynamic and fast instance segmentation. Proc Adv Neural Inf Process Sys (NeurIPS)
Wang X, Kong T, Shen C, Jiang Y, Li L (2020) SOLO: segmenting objects by locations. In: Proc Eur Conf Computer Vision (ECCV)
Fang Y, Yang S, Wang X, Li Y, Fang C, Shan Y, Feng B, Liu W (2021) Instances as queries. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pp 6910–6919
Vu T, Haeyong K, Yoo CD (2021) Scnet: training inference sample consistency for instance segmentation. In: AAAI
Qiao S, Chen L-C, Yuille A (2021) Detectors: detecting objects with recursive feature pyramid and switchable atrous convolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10213–10224
Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu J, Zhang Z, Cheng D, Zhu C, Cheng T, Zhao Q, Li B, Lu X, Zhu R, Wu Y, Dai J, Wang J, Shi J, Ouyang W, Loy CC, Lin D (2019) MMDetection: open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16x16 words: transformers for image recognition at scale. ICLR
Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg AC, Lo W-Y et al (2023) Segment anything. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4015–4026
Acknowledgements
The authors would like to express their appreciation to the referees for their helpful comments and suggestions. This work was supported in part by the National Natural Science Foundation of China (Grant nos. 62373324 and 62271448), and in part by the Zhejiang Provincial Natural Science Foundation of China (Grant no. LGF22F030016).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by [Qi Chen] and [Haigen Hu]. The first draft of the manuscript was written by [Huihuang Zhang] and [Qiu Guan]. The grammar and format are edited by [Qianwei Zhou] and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethical and Informed Consent for Data Used
Authors have cited any publicly available data on which the conclusions of the paper rely.
Competing of Interest
Authors declare no competing interests that are directly or indirectly related to the work submitted for publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, Q., Zhang, H., Zhou, Q. et al. An anchor-free instance segmentation method for cells based on mask contour. Appl Intell 55, 111 (2025). https://doi.org/10.1007/s10489-024-06004-w
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06004-w