Abstract
Multiplex-brightfield immunohistochemistry imaging (MPx) enables quantification of various biomarkers in tissue while retaining morphological and spatial information. One of the critical challenges of MPx is the complexity of visually inspection-based assessment of multiple stain intensities when they are co-localized in a cell. Consequently, it requires digital unmixing methods to separate multiple staining components and remix individual staining elements together with counterstain to become synthetic singleplex images (SPx). Unmixing MPx images becomes even more challenging when more than three biomarkers co-localized. Conventional unmixing methods e.g.,color-deconvolution or Nonnegative-Matrix-Factorization are limited and error prone when separating staining intensities of co-localized biomarkers in membrane or nuclear-subcellular compartments. Here, we exploit advances in generative-adversarial networks (GANs) based on unpaired image-to-image translation (CycleGAN) to generate synthetic SPx from MPx for pathologists to read and score. Three tonsil tissues with a total of 36 wholeslide images, stained with CD3, Bcl2, and CD8 using chromogenic detection, were used for training and evaluating our framework. Adjacent SPx were used to evaluate the visual quality of our synthetic SPx images with the following experiments: 1) performed perceptual studies or “real-vs.-fake” based on Amazon-Mechanical Turk (AMT) with pathologist observers, where results showed synthetic SPx were indistinguishable from real-adjacent SPx; and 2) evaluated whether our synthetic SPx were realistic to be scored for intensity. Results showed similarity scores of 0.96, 0.96, and 0.97 overall intensity for each synthetic SPx, respectively. Our framework provides alternative methods to virtually unmix the stains in order to accurately and efficiently generate synthetic SPx images from MPx tissue slides. This can bring confidence and opportunities for MPx in-vitro diagnostics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Morrison, L.E., Lefever, M.R., Behman, L.J., Leibold, T., Roberts, E.A., Horchner, U.B., et al.: Brightfield multiplex immunohistochemistry with multispectral imaging. Lab. Invest. 100, 1124–1136 (2020)
Tan, W.C.C., Nerurkar, S.N., Cai, H.Y., Ng, H.H.M., Wu, D., Wee, Y.T.F., et al.: Overview of multiplex im-munohistochemistry/immunofluorescence techniques in the era of cancer immunotherapy. Cancer Commun. 40, 135–153 (2020)
Joensuu, K., Leidenius, M., Kero, M., Andersson, L.C., Horwitz, K.B., Heikkilä, P.: ER, PR, HER2, Ki-67 and CK5 in early and late relapsing breast cancer-reduced CK5 expression in metastases. Breast Cancer (Auckl). 7, 23–34 (2013)
Ruifrok, A.C., Johnston, D.A.: Quantification of histochemical staining by color deconvolution. J. Chem. Inf. Model 53, 1689–1699 (2013)
Zhang, J., Zhang, X., Jiao, L.: Sparse Nonnegative Matrix Factorization for Hyperspectral Unmixing Based on Endmember Independence and Spatial Weighted Abundance. Remote Sens (Basel), vol. 13 (2021)
Bouteldja, N., Hölscher, D.L., Bülow, R.D., Roberts, I.S.D., Coppo, R., Boor, P.: Tackling stain variability using CycleGAN-based stain augmentation. J. Pathol. Inform. 13, 100140 (2022)
Kapil, A., , et al.: DASGAN -- Joint Do-main Adaptation and Segmentation for the Analysis of Epithelial Regions in Histopathology PD-L1 Images. Comput. Vis. Pattern Recogn (2019)
Gadermayr, M., Gupta, L., Klinkhammer, B.M., Boor, P., Merhof, D.: Unsupervisedly training GANs for segmenting digital pathology with automatically generated annotations. In: ISBI (2019)
Gupta, L., Klinkhammer, B.M., Boor, P., Merhof, D., Gadermayr, M.: GAN-based image enrichment in digital pathology boosts segmentation accuracy. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, pp. 631–639. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_70
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-Image Translation with Conditional Adversarial Networks (2016)
Zhao, Y., Ruihai, Wu., Dong, H.: Unpaired image-to-image translation using adversarial consistency loss. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX, pp. 800–815. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_46
Zhang, W., Hubbard, A., Jones, T., Racolta, A., Bhaumik, S., Cummins, N., et al.: Fully automated 5-plex fluorescent immunohistochemistry with tyramide signal amplification and same species anti-bodies. Lab. Invest. 97, 873–885 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lorsakul, A. et al. (2023). Synthetic Singleplex-Image Generation in Multiplex-Brightfield Immunohistochemistry Digital Pathology Using Deep Generative Models. In: Wolterink, J.M., Svoboda, D., Zhao, C., Fernandez, V. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2023. Lecture Notes in Computer Science, vol 14288. Springer, Cham. https://doi.org/10.1007/978-3-031-44689-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-44689-4_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44688-7
Online ISBN: 978-3-031-44689-4
eBook Packages: Computer ScienceComputer Science (R0)
