GeNSeg-Net: A General Segmentation Framework for Any Nucleus in Immunohistochemistry Images
Authors: 
Siyuan Xu, Guannan Li, Haofei Song, Jiansheng Wang, Yan Wang, Qingli LiAuthors Info & Claims
Siyuan Xu, Guannan Li, Haofei Song, Jiansheng Wang, Yan Wang, Qingli LiAuthors Info & ClaimsPages 4475 - 4484
Abstract
Immunohistochemistry (IHC) plays a crucial role in understanding disease mechanisms, diagnosing pathology and guiding treatment decisions. The precise analysis heavily depends on accurate nucleus segmentation. However, segmentation is challenging due to significant inter- and intra-nucleus variability in morphology and distribution, stemming from inherent characteristics, imaging techniques, tissue differences and other factors. While current deep learning-based methods have shown promising results, their generalization performance is limited, inevitably requiring specific training data. To address the problem, we propose a novel Ge neral framework for Nucleus Seg mentation in IHC images (GeNSeg-Net). GeNSeg-Net effectively segments nuclei across diverse tissue types and imaging techniques with high variability using a small subset for training. It comprises an enhancement model and a segmentation model. Initially, all nuclei are enhanced to a uniform morphology with distinct features by the enhancement model through generation. The subsequent segmentation task is thereby simplified, leading to higher accuracy. We design a lightweight generator and discriminator to improve both enhancement quality and computational efficiency. Extensive experiments demonstrate the effectiveness of each component within GeNSeg-Net. Compared to existing methods, GeNSeg-Net achieves state-of-the-art (SOTA) segmentation accuracy and generalization performance on both private and public datasets, while maintaining highly competitive processing speed. Code is available at https://github.com/SikangSHU/GeNSeg-Net.
References
[1]
Navid Alemi Koohbanani, Mostafa Jahanifar, Ali Gooya, and Nasir Rajpoot. 2019. Nuclear instance segmentation using a proposal-free spatially aware deep learning framework. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 622--630.
[2]
T Bonacho, Francisco Rodrigues, and Joana Liberal. 2020. Immunohistochemistry for diagnosis and prognosis of breast cancer: A review. Biotechnic & Histochemistry, Vol. 95, 2 (2020), 71--91.
[3]
Yuri Y Boykov and M-P Jolly. 2001. Interactive graph cuts for optimal boundary & region segmentation of objects in ND images. In Proceedings of the IEEE International Conference on Computer Vision, Vol. 1. IEEE, 105--112.
[4]
Juan C Caicedo, Allen Goodman, Kyle W Karhohs, Beth A Cimini, Jeanelle Ackerman, Marzieh Haghighi, CherKeng Heng, Tim Becker, Minh Doan, Claire McQuin, et al. 2019. Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nature Methods, Vol. 16, 12 (2019), 1247--1253.
[5]
Juan C Caicedo, Jonathan Roth, Allen Goodman, Tim Becker, Kyle W Karhohs, Matthieu Broisin, Csaba Molnar, Claire McQuin, Shantanu Singh, Fabian J Theis, et al. 2019. Evaluation of deep learning strategies for nucleus segmentation in fluorescence images. Cytometry Part A, Vol. 95, 9 (2019), 952--965.
[6]
Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S Yu, and Lichao Sun. 2023. A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt. ArXiv Preprint ArXiv:2303.04226 (2023).
[7]
Hao Chen, Xiaojuan Qi, Lequan Yu, and Pheng-Ann Heng. 2016. DCAN: Deep contour-aware networks for accurate gland segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2487--2496.
[8]
Junzhou Chen, Qian Huang, Yulin Chen, Linyi Qian, and Chengyuan Yu. 2023. Enhancing nucleus segmentation with haru-net: a hybrid attention based residual u-blocks network. ArXiv Preprint ArXiv:2308.03382 (2023).
[9]
Qi Chen, Wei Huang, Xiaoyu Liu, Jiacheng Li, and Zhiwei Xiong. 2023. PCTrans: Position-Guided Transformer with Query Contrast for Biological Instance Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3903--3912.
[10]
Shengcong Chen, Changxing Ding, Minfeng Liu, Jun Cheng, and Dacheng Tao. 2023. CPP-net: Context-aware polygon proposal network for nucleus segmentation. IEEE Transactions on Image Processing, Vol. 32 (2023), 980--994.
[11]
Shengcong Chen, Changxing Ding, and Dacheng Tao. 2020. Boundary-assisted region proposal networks for nucleus segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 279--288.
[12]
Kin-Hoe Chow, Rachel E Factor, and Katharine S Ullman. 2012. The nuclear envelope environment and its cancer connections. Nature Reviews Cancer, Vol. 12, 3 (2012), 196--209.
[13]
Leandro Luongo De Matos, Damila Cristina Trufelli, Maria Graciela Luongo De Matos, and Maria Aparecida da Silva Pinhal. 2010. Immunohistochemistry as an important tool in biomarkers detection and clinical practice. Biomarker Insights, Vol. 5 (2010), BMI--S2185.
[14]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. ArXiv Preprint ArXiv:2010.11929 (2020).
[15]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems, Vol. 27.
[16]
Simon Graham, Quoc Dang Vu, Shan E Ahmed Raza, Ayesha Azam, Yee Wah Tsang, Jin Tae Kwak, and Nasir Rajpoot. 2019. Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Medical Image Analysis, Vol. 58 (2019), 101563.
[17]
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision. 2961--2969.
[18]
Fabian Hörst, Moritz Rempe, Lukas Heine, Constantin Seibold, Julius Keyl, Giulia Baldini, Selma Ugurel, Jens Siveke, Barbara Grünwald, Jan Egger, et al. 2024. Cellvit: Vision transformers for precise cell segmentation and classification. Medical Image Analysis, Vol. 94 (2024), 103143.
[19]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1125--1134.
[20]
Yifan Jiang, Shiyu Chang, and Zhangyang Wang. 2021. Transgan: Two pure transformers can make one strong gan, and that can scale up. In Advances in Neural Information Processing Systems, Vol. 34. 14745--14758.
[21]
Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollár. 2019. Panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9404--9413.
[22]
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4015--4026.
[23]
Neeraj Kumar, Ruchika Verma, Sanuj Sharma, Surabhi Bhargava, Abhishek Vahadane, and Amit Sethi. 2017. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Transactions on Medical Imaging, Vol. 36, 7 (2017), 1550--1560.
[24]
Shyam Lal, Devikalyan Das, Kumar Alabhya, Anirudh Kanfade, Aman Kumar, and Jyoti Kini. 2021. NucleiSegNet: Robust deep learning architecture for the nuclei segmentation of liver cancer histopathology images. Computers in Biology and Medicine, Vol. 128 (2021), 104075.
[25]
Dongnan Liu, Donghao Zhang, Yang Song, Heng Huang, and Weidong Cai. 2021. Panoptic feature fusion net: a novel instance segmentation paradigm for biomedical and biological images. IEEE Transactions on Image Processing, Vol. 30 (2021), 2045--2059.
[26]
Vebjorn Ljosa, Katherine L Sokolnicki, and Anne E Carpenter. 2012. Annotated high-throughput microscopy image sets for validation. Nature Methods, Vol. 9, 7 (2012), 637--637.
[27]
Cheng Lu, David Romo-Bucheli, Xiangxue Wang, Andrew Janowczyk, Shridar Ganesan, Hannah Gilmore, David Rimm, and Anant Madabhushi. 2018. Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers. Laboratory Investigation, Vol. 98, 11 (2018), 1438--1448.
[28]
Zhi Lu, Gustavo Carneiro, and Andrew P Bradley. 2015. An improved joint optimization of multiple level set functions for the segmentation of overlapping cervical cells. IEEE Transactions on Image Processing, Vol. 24, 4 (2015), 1261--1272.
[29]
Norberto Malpica, Carlos Ortiz De Solórzano, Juan José Vaquero, Andrés Santos, Isabel Vallcorba, José Miguel García-Sagredo, and Francisco Del Pozo. 1997. Applying watershed algorithms to the segmentation of clustered nuclei. Cytometry: The Journal of the International Society for Analytical Cytology, Vol. 28, 4 (1997), 289--297.
[30]
Csaba Molnar, Ian H Jermyn, Zoltan Kato, Vesa Rahkama, Päivi Östling, Piia Mikkonen, Vilja Pietiäinen, and Peter Horvath. 2016. Accurate morphology preserving segmentation of overlapping cells based on active contours. Scientific Reports, Vol. 6, 1 (2016), 32412.
[31]
Hirohisa Oda, Holger R Roth, Kosuke Chiba, Jure Sokolić, Takayuki Kitasaka, Masahiro Oda, Akinari Hinoki, Hiroo Uchida, Julia A Schnabel, and Kensaku Mori. 2018. BESNet: Boundary-enhanced segmentation of cells in histopathological images. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 228--236.
[32]
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. 2016. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2536--2544.
[33]
Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. ArXiv Preprint ArXiv:1511.06434 (2015).
[34]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 234--241.
[35]
Uwe Schmidt, Martin Weigert, Coleman Broaddus, and Gene Myers. 2018. Cell detection with star-convex polygons. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 265--273.
[36]
David Stutz, Alexander Hermans, and Bastian Leibe. 2018. Superpixels: An evaluation of the state-of-the-art. Computer Vision and Image Understanding, Vol. 166 (2018), 1--27.
[37]
Wei Chang Colin Tan, Sanjna Nilesh Nerurkar, Hai Yun Cai, Harry Ho Man Ng, Duoduo Wu, Yu Ting Felicia Wee, Jeffrey Chun Tatt Lim, Joe Yeong, and Tony Kiat Hon Lim. 2020. Overview of multiplex immunohistochemistry/immunofluorescence techniques in the era of cancer immunotherapy. Cancer Communications, Vol. 40, 4 (2020), 135--153.
[38]
Afaf Tareef, Yang Song, Heng Huang, Dagan Feng, Mei Chen, Yue Wang, and Weidong Cai. 2018. Multi-pass fast watershed for accurate segmentation of overlapping cervical cells. IEEE Transactions on Medical Imaging, Vol. 37, 9 (2018), 2044--2059.
[39]
Janis M Taube, Guray Akturk, Michael Angelo, Elizabeth L Engle, Sacha Gnjatic, Shirley Greenbaum, Noah F Greenwald, Cyrus V Hedvat, Travis J Hollmann, Jonathan Juco, et al. 2020. The society for immunotherapy in cancer statement on best practices for multiplex immunohistochemistry (IHC) and immunofluorescence (IF) staining and validation. Journal for Immunotherapy of Cancer, Vol. 8, 1 (2020).
[40]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30.
[41]
Tomas Vicar, Jan Balvan, Josef Jaros, Florian Jug, Radim Kolar, Michal Masarik, and Jaromir Gumulec. 2019. Cell segmentation methods for label-free contrast microscopy: review and comprehensive comparison. BMC Bioinformatics, Vol. 20 (2019), 1--25.
[42]
Carolina Wählby, I-M Sintorn, Fredrik Erlandsson, Gunilla Borgefors, and Ewert Bengtsson. 2004. Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections. Journal of Microscopy, Vol. 215, 1 (2004), 67--76.
[43]
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7794--7803.
[44]
Xiaolong Wang and Abhinav Gupta. 2016. Generative image modeling using style and structure adversarial networks. In European Conference on Computer Vision. Springer, 318--335.
[45]
Mark Winter, Walter Mankowski, Eric Wait, Edgar Cardenas De La Hoz, Angeline Aguinaldo, and Andrew R Cohen. 2018. Separating touching cells using pixel replicated elliptical shape models. IEEE Transactions on Medical Imaging, Vol. 38, 4 (2018), 883--893.
[46]
Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision. 3--19.
[47]
Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and Hong Lin. 2023. Ai-generated content (aigc): A survey. ArXiv Preprint ArXiv:2304.06632 (2023).
[48]
Xiaodong Yang, Houqiang Li, and Xiaobo Zhou. 2006. Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and Kalman filter in time-lapse microscopy. IEEE Transactions on Circuits and Systems I: Regular Papers, Vol. 53, 11 (2006), 2405--2414.
[49]
Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. 2019. Self-attention generative adversarial networks. In International Conference on Machine Learning. PMLR, 7354--7363.
[50]
Bingchao Zhao, Xin Chen, Zhi Li, Zhiwen Yu, Su Yao, Lixu Yan, Yuqian Wang, Zaiyi Liu, Changhong Liang, and Chu Han. 2020. Triple U-net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation. Medical Image Analysis, Vol. 65 (2020), 101786.
[51]
Yanning Zhou, Omer Fahri Onder, Qi Dou, Efstratios Tsougenis, Hao Chen, and Pheng-Ann Heng. 2019. Cia-net: Robust nuclei instance segmentation with contour-aware information aggregation. In Information Processing in Medical Imaging. Springer, 682--693.
Index Terms
- GeNSeg-Net: A General Segmentation Framework for Any Nucleus in Immunohistochemistry Images
Recommendations
Segmentation of complex nucleus configurations in biological images
This paper presents a new segmentation method of complex nucleus configurations. The specificity of this work is to introduce prior information about the usual shape of cells nuclei, in order to optimize the selection of markers from which the flooding ...
Segmentation of cervical cell nuclei in high-resolution microscopic images
Highlights A system for semi-automatic processing of full-resolution whole-slide scans. A new algorithm for segmenting the nuclei under control of the expert user. Data storage and interaction of technical and medical experts is facilitated. Open source ...
ASW-Net: A Deep Learning-based Tool for Cell Nucleus Segmentation of Fluorescence Microscopy
ICBBS '21: Proceedings of the 2021 10th International Conference on Bioinformatics and Biomedical ScienceNucleus segmentation of fluorescence microscopy is a critical step in quantifying measurements in cell biology. Automatic and accurate nucleus segmentation has powerful applications in analyzing intrinsic characterization in nucleus morphology. However, ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 64Total Downloads
- Downloads (Last 12 months)64
- Downloads (Last 6 weeks)48
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in