Abstract
Numerous deep learning based methods have been developed for nuclei segmentation for H &E images and have achieved close to human performance. However, direct application of such methods to another modality of images, such as Immunohistochemistry (IHC) images, may not achieve satisfactory performance. Thus, we developed a Generative Adversarial Network (GAN) based approach to translate an IHC image to an H &E image while preserving nuclei location and morphology and then apply pre-trained nuclei segmentation models to the virtual H &E image. We demonstrated that the proposed methods work better than several baseline methods including direct application of state of the art nuclei segmentation methods such as Cellpose and HoVer-Net, trained on H &E and a generative method, DeepLIIF, using two public IHC image datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abdolhoseini, M., Kluge, M.G., Walker, F.R., Johnson, S.J.: Segmentation of heavily clustered nuclei from histopathological images. Sci. Rep. 9(1), 1–13 (2019)
Bayramoglu, N., Kaakinen, M., Eklund, L., Heikkila, J.: Towards virtual H &E staining of hyperspectral lung histology images using conditional generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 64–71 (2017)
Caicedo, J.C., et al.: Nucleus segmentation across imaging experiments: the 2018 data science bowl. Nat. Methods 16(12), 1247–1253 (2019)
Chen, M., Artières, T., Denoyer, L.: Unsupervised object segmentation by redrawing. Adv. Neural Inf. Process. Syst. 32 (2019)
Coelho, L.P., Shariff, A., Murphy, R.F.: Nuclear segmentation in microscope cell images: a hand-segmented dataset and comparison of algorithms. In: 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 518–521. IEEE (2009)
Di Cataldo, S., Ficarra, E., Acquaviva, A., Macii, E.: Automated segmentation of tissue images for computerized IHC analysis. Comput. Methods Programs Biomed. 100(1), 1–15 (2010)
Ghahremani, P., et al.: Deep learning-inferred multiplex immunofluorescence for immunohistochemical image quantification. Nat. Mach. Intell. 4(4), 401–412 (2022)
Ghahremani, P., et al.: Deepliif (2022). https://github.com/nadeemlab/DeepLIIF
Glass, B., et al.: 821 machine learning models can quantify cd8 positivity in lymphocytes in melanoma clinical trial samples. J. Immunother. Cancer 9(Suppl 2), A859–A859 (2021)
Graham, S., et al.: Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019)
Gutman, D.A., et al.: The digital slide archive: a software platform for management, integration, and analysis of histology for cancer research. Cancer Res. 77(21), e75–e78 (2017). https://doi.org/10.1158/0008-5472.CAN-17-0629
Han, L., Yin, Z.: Unsupervised network learning for cell segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 282–292. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_27
Hatipoglu, N., Bilgin, G.: Cell segmentation in histopathological images with deep learning algorithms by utilizing spatial relationships. Med. Biol. Eng. Comput. 55(10), 1829–1848 (2017). https://doi.org/10.1007/s11517-017-1630-1
Ho, M.Y., Chapman, V., Ali, Z., Graham, S., Vu, Q.D.: Hover-net (2022). https://github.com/vqdang/hover_net
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134 (2017)
Kaplan, K.: Quantifying IHC data from whole slide images is paving the way toward personalized medicine. MLO Med. Lab. Obs. 47, 20–21 (2015)
Kim, J., Kim, M., Kang, H., Lee, K.H.: U-gat-it: unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=BJlZ5ySKPH
Kumar, N., Verma, R., Sharma, S., Bhargava, S., Vahadane, A., Sethi, A.: A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 36(7), 1550–1560 (2017)
Mahbod, A., et al.: CryoNuSeg: a dataset for nuclei instance segmentation of cryosectioned H &E-stained histological images. Comput. Biol. Med. 132, 104349 (2021)
Naylor, P., Laé, M., Reyal, F., Walter, T.: Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans. Med. Imaging 38(2), 448–459 (2018)
Rivenson, Y., de Haan, K., Wallace, W.D., Ozcan, A.: Emerging advances to transform histopathology using virtual staining. BME Front. 2020 (2020)
Rivenson, Y., et al.: Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng. 3(6), 466–477 (2019)
Schmidt, U., Weigert, M.: Stardist (2022). https://github.com/stardist/stardist
Schmidt, U., Weigert, M., Broaddus, C., Myers, G.: Cell detection with star-convex polygons. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2018–21st International Conference, Granada, Spain, 16–20 September 2018, Proceedings, Part II, pp. 265–273 (2018). https://doi.org/10.1007/978-3-030-00934-2_30
Stringer, C.: Cellpose (2022). https://github.com/MouseLand/cellpose
Stringer, C., Wang, T., Michaelos, M., Pachitariu, M.: Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18(1), 100–106 (2021)
Swiderska-Chadaj, Z., et al.: Learning to detect lymphocytes in immunohistochemistry with deep learning. Med. Image Anal. 58, 101547 (2019)
Xu, Z., Moro, C.F., Bozóky, B., Zhang, Q.: Gan-based virtual re-staining: a promising solution for whole slide image analysis. arXiv preprint arXiv:1901.04059 (2019)
Yang, L., et al.: NuSeT: a deep learning tool for reliably separating and analyzing crowded cells. PLoS Comput. Biol. 16(9), e1008193 (2020)
Zhang, Y., de Haan, K., Rivenson, Y., Li, J., Delis, A., Ozcan, A.: Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci. Appl. 9(1), 1–13 (2020)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Trullo, R., Bui, QA., Tang, Q., Olfati-Saber, R. (2022). Image Translation Based Nuclei Segmentation for Immunohistochemistry Images. In: Mukhopadhyay, A., Oksuz, I., Engelhardt, S., Zhu, D., Yuan, Y. (eds) Deep Generative Models. DGM4MICCAI 2022. Lecture Notes in Computer Science, vol 13609. Springer, Cham. https://doi.org/10.1007/978-3-031-18576-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-18576-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18575-5
Online ISBN: 978-3-031-18576-2
eBook Packages: Computer ScienceComputer Science (R0)
