Realistic cross-domain microscopy via conditional generative adversarial networks: converting immunofluorescence to hematoxylin and eosin

Gouthamrajan Nadarajan; Scott Doyle

doi:10.1117/12.2549842

16 March 2020 Realistic cross-domain microscopy via conditional generative adversarial networks: converting immunofluorescence to hematoxylin and eosin

Gouthamrajan Nadarajan, Scott Doyle

Author Affiliations +

Proceedings Volume 11320, Medical Imaging 2020: Digital Pathology; 113200S (2020) https://doi.org/10.1117/12.2549842
Event: SPIE Medical Imaging, 2020, Houston, Texas, United States

Abstract

Hematoxylin and Eosin (H&E) is a widely-used stain for diagnosis and prognosis in clinical pathology; it is a non-specific stain, binding to all cell types. Immunofluorescent (IF) staining is highly specific, binding only to targeted proteins in a sample to identify specific cellular and sub-cellular structures. IF images are costlier and more technically difficult compared with H&E, so are rarely used in routine clinical workup, but can be used for identification of diagnostically significant cell types. In previous work, we used registered IF and H&E images to generate class labels for training a deep learning H&E segmentation algorithm. In this work, we leverage this dataset to train a Conditional Generative Adversarial Network (cGAN) to generate realistic-looking H&E images from IF images stained for DAPI and ribosomal S6. Using these generated images, we trained a semantic segmentation algorithm to identify nuclei, cytoplasm, and membrane classes by thresholding the original IF stains for use as class labels on the generated H&E. The trained classifier was then used to segment a holdout dataset of real H&E images. We found that the semantic segmentation models trained on the generated H&E images (Dice score: 0.539) performed similarly to models trained on real H&E (Dice score: 0.503), suggesting that cGAN generated samples can be used as a viable training set for deep learning models that are intended to be applied on real H&E data.

Conference Presentation

Citation Download Citation

Gouthamrajan Nadarajan and Scott Doyle "Realistic cross-domain microscopy via conditional generative adversarial networks: converting immunofluorescence to hematoxylin and eosin", Proc. SPIE 11320, Medical Imaging 2020: Digital Pathology, 113200S (16 March 2020); https://doi.org/10.1117/12.2549842

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available