Hengxu Zhang
ABSTRACT
The detection and segmentation of cell nuclei in whole slide tissue images plays an important role in disease diagnosis and treatment. Automatic detection and segmentation of nuclei is very challenging due to high nuclei density, low contrast, overlapping and adhesion between cells. Recently, Transformer-based object detection and instance segmentation methods have made great progress on traditional computer vision datasets. These Transformer-based approaches are effective by removing the need for many hand-crafted components in the network, and post-processing steps like non-maximum suppression (NMS). However, such approaches consume tremendous amount of memory in detection and segmentation of cell nuclei in whole slide tissue images due to large number of cells in the images. Also, those methods may suffer from inferior performance on small cell instances. Inspired by Deformable DETR which makes use of small set of key sampling points in the attention module to reduce the computation and the usage of multi-scale feature maps, we propose an efficient Transformer-based approach for joint nuclei detection and segmentation in whole slide tissue images. Specifically, In Transformer decoder, it directly outputs the object detection results and instance segmentation masks. We evaluate our proposed method on the public MoNuSeg dataset, and the experimental results show that our method achieves promising performance on nuclei detection and segmentation.
- Rolf Adams and Leanne Bischof. 1994. Seeded region growing. IEEE Transactions on pattern analysis and machine intelligence 16, 6(1994), 641–647.Google Scholar
Digital Library
- Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934(2020).Google Scholar
- Daniel Bolya, Chong Zhou, Fanyi Xiao, and Yong Jae Lee. 2019. Yolact: Real-time instance segmentation. In Proceedings of the IEEE/CVF international conference on computer vision. 9157–9166.Google ScholarCross Ref
- P Bonnin, J Blanc Talon, J C Hayot, and Bertrand Zavidovique. 1990. A new edge point/region cooperative segmentation deduced from a 3d scene reconstruction application. In Applications of Digital Image Processing XII, Vol. 1153. SPIE, 579–591.Google Scholar
- Zhaowei Cai and Nuno Vasconcelos. 2018. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6154–6162.Google ScholarCross Ref
- Jiale Cao, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan, Yanwei Pang, and Ling Shao. 2020. D2det: Towards high quality object detection and instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11485–11494.Google ScholarCross Ref
- Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. 2020. End-to-End Object Detection with Transformers. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I. 213–229.Google Scholar
- Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, 2019. Hybrid task cascade for instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4974–4983.Google ScholarCross Ref
- Yuxin Fang, Shusheng Yang, Xinggang Wang, Yu Li, Chen Fang, Ying Shan, Bin Feng, and Wenyu Liu. 2021. Instances as queries. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 6910–6919.Google ScholarCross Ref
- Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision. 1440–1448.Google ScholarDigital Library
- Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 315–323.Google Scholar
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision. 2961–2969.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.Google ScholarCross Ref
- Neeraj Kumar, Ruchika Verma, Deepak Anand, Yanning Zhou, Omer Fahri Onder, Efstratios Tsougenis, Hao Chen, Pheng-Ann Heng, Jiahui Li, Zhiqiang Hu, 2019. A multi-organ nucleus segmentation challenge. IEEE transactions on medical imaging 39, 5 (2019), 1380–1391.Google Scholar
- Hei Law and Jia Deng. 2018. Cornernet: Detecting objects as paired keypoints. In Proceedings of the European conference on computer vision (ECCV). 734–750.Google ScholarDigital Library
- Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980–2988.Google ScholarCross Ref
- Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740–755.Google ScholarCross Ref
- Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. In European conference on computer vision. Springer, 21–37.Google ScholarCross Ref
- Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431–3440.Google ScholarCross Ref
- Ilya Loshchilov and Frank Hutter. 2018. Decoupled Weight Decay Regularization. In International Conference on Learning Representations.Google Scholar
- Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. 2016. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV). IEEE, 565–571.Google Scholar
- Mohammad Hasanzadeh Mofrad, Sana Sadeghi, Alireza Rezvanian, and Mohammad Reza Meybodi. 2015. Cellular edge detection: Combining cellular automata and cellular learning automata. AEU-International Journal of Electronics and Communications 69, 9(2015), 1282–1290.Google ScholarCross Ref
- Ethan H Nguyen, Haichun Yang, Ruining Deng, Yuzhe Lu, Zheyu Zhu, Joseph T Roland, Le Lu, Bennett A Landman, Agnes B Fogo, and Yuankai Huo. 2021. Circle Representation for Medical Object Detection. IEEE transactions on medical imaging 41, 3 (2021), 746–754.Google Scholar
- Joseph Redmon and Ali Farhadi. 2017. YOLO9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7263–7271.Google ScholarCross Ref
- Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767(2018).Google Scholar
- S Ren, K He, R Girshick, and J Sun. 2016. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 6(2016), 1137–1149.Google ScholarDigital Library
- Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. 2019. Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 658–666.Google ScholarCross Ref
- Guanglu Song, Yu Liu, and Xiaogang Wang. 2020. Revisiting the sibling head in object detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11563–11572.Google ScholarCross Ref
- Zhi Tian, Chunhua Shen, and Hao Chen. 2020. Conditional convolutions for instance segmentation. In European conference on computer vision. Springer, 282–298.Google ScholarDigital Library
- Alain Tremeau and Nathalie Borel. 1997. A region growing and merging algorithm to color segmentation. Pattern recognition 30, 7 (1997), 1191–1203.Google Scholar
- Luc Vincent and Pierre Soille. 1991. Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis & Machine Intelligence 13, 06(1991), 583–598.Google ScholarDigital Library
- Xinlong Wang, Tao Kong, Chunhua Shen, Yuning Jiang, and Lei Li. 2020. Solo: Segmenting objects by locations. In European Conference on Computer Vision. Springer, 649–665.Google ScholarDigital Library
- Xiaodong Yu, Dahu Shi, Xing Wei, Ye Ren, Tingqun Ye, and Wenming Tan. 2022. Soit: Segmenting objects with instance-aware transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 3188–3196.Google ScholarCross Ref
- Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. 2019. Objects as Points. arXiv e-prints (2019), arXiv–1904.Google Scholar
- Xingyi Zhou, Jiacheng Zhuo, and Philipp Krahenbuhl. 2019. Bottom-up object detection by grouping extreme and center points. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 850–859.Google ScholarCross Ref
- Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. 2020. Deformable DETR: Deformable Transformers for End-to-End Object Detection. In International Conference on Learning Representations.Google Scholar
Index Terms
- An Efficient Transformer-based Approach for Joint Nuclei Detection and Segmentation in Whole Slide Tissue Images
Recommendations
Simultaneous Detection and Segmentation of Cell Nuclei based on Convolutional Neural Network
ISICDM 2018: Proceedings of the 2nd International Symposium on Image Computing and Digital MedicineIn this paper, we mainly focus on automatic detection and segmentation of cell nuclei in histopathology images. Though some methods have been presented to solve these issues, there is scope for efficiency and performance to improve. We propose an end-to-...
Segmentation of cell nuclei in fluorescence microscopy images
Accurate segmentation of cells in fluorescence microscopy images plays a key role in high-throughput applications such as quantification of protein expression and the study of cell function. In this paper, an integrated framework consisting of a new ...
Comments