Abstract
In Whole Slide Image (WSI) analysis, detecting nuclei sub-types such as Tumor Infiltrating Lymphocytes (TILs) which are a primary bio-marker for cancer diagnosis, is an important yet challenging task. Though several conventional methods have been proposed and applied to target user’s nuclei sub-types (e.g., TILs), they often fail to detect subtle differences between instances due to similar morphology across sub-types. To address this, we propose a novel decoupled segmentation architecture that leverages point annotations in a weakly-supervised manner to adapt to the nuclei sub-type. Our design consists of an encoder for feature extraction, a boundary regressor that learns prior knowledge from nuclei boundary masks, and a point detector that predicts the center positions of nuclei, respectively. Moreover, employing a frozen pre-trained nuclei segmenter facilitates easier adaptation to TILs segmentation via fine-tuning, while learning a decoupled point detector. To demonstrate the effectiveness of our approach, we evaluated on an in-house Melanoma TIL dataset, and report significant improvements over a state-of-the-art weakly-supervised TILs segmentation method, including conventional approaches based on pseudo-label construction.
S. Nam and M. Knag—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amgad, M., et al.: Joint region and nucleus segmentation for characterization of tumor infiltrating lymphocytes in breast cancer. In: Medical Imaging 2019: Digital Pathology, vol. 10956, p. 109560M. International Society for Optics and Photonics (2019)
Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)
Deng, Y., Manjunath, B.S., Shin, H.: Color image segmentation. In: Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No. PR00149), vol. 2, pp. 446–451. IEEE (1999)
Gardeux, V., David, F.P., Shajkofci, A., Schwalie, P.C., Deplancke, B.: ASAP: a web-based platform for the analysis and interactive visualization of single-cell RNA-seq data. Bioinformatics 33(19), 3123–3125 (2017)
Graham, S., et al.: Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019)
Heckbert, P.S.: A seed fill algorithm. Graph. Gems 275, 721–722 (1990)
Khoreva, A., Benenson, R., Hosang, J., Hein, M., Schiele, B.: Simple does it: weakly supervised instance and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 876–885 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kumar, N., et al.: A multi-organ nucleus segmentation challenge. IEEE Trans. Med. Imaging 39(5), 1380–1391 (2019)
Kumar, N., Verma, R., Sharma, S., Bhargava, S., Vahadane, A., Sethi, A.: A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 36(7), 1550–1560 (2017)
Lin, D., Dai, J., Jia, J., He, K., Sun, J.: Scribblesup: scribble-supervised convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3159–3167 (2016)
Luna, M., Kwon, M., Park, S.H.: Precise separation of adjacent nuclei using a Siamese neural network. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 577–585. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_64
Qu, H., et al.: Weakly supervised deep nuclei segmentation using points annotation in histopathology images. In: International Conference on Medical Imaging with Deep Learning, pp. 390–400. PMLR (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Tian, K., et al.: Weakly-supervised nucleus segmentation based on point annotations: a coarse-to-fine self-stimulated learning strategy. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 299–308. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_29
Yao, K., Huang, K., Sun, J., Hussain, A., Jude, C.: PointNu-Net: simultaneous multi-tissue histology nuclei segmentation and classification in the clinical wild. arXiv preprint arXiv:2111.01557 (2021)
Yoo, I., Yoo, D., Paeng, K.: PseudoEdgeNet: nuclei segmentation only with point annotations. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 731–739. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_81
Acknowledgment
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIT) (No. 2019R1C1C1008727), and Smart Health Care Program funded by the Korean National Police Agency (220222M01).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nam, S. et al. (2022). Weakly-Supervised TILs Segmentation Based on Point Annotations Using Transfer Learning with Point Detector and Projected-Boundary Regressor. In: Rekik, I., Adeli, E., Park, S.H., Cintas, C. (eds) Predictive Intelligence in Medicine. PRIME 2022. Lecture Notes in Computer Science, vol 13564. Springer, Cham. https://doi.org/10.1007/978-3-031-16919-9_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-16919-9_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16918-2
Online ISBN: 978-3-031-16919-9
eBook Packages: Computer ScienceComputer Science (R0)
