Skip to main content
SpringerLink
Log in

Staining condition visualization in digital histopathological whole-slide images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Staining condition is one of the essential properties in digital pathology for developing computer-aided diagnosis (CAD) systems; however, it is challenging to analyze the staining condition of giga-pixel whole-slide images (WSIs) due to the high data volume. In this study, we proposed an intuitive method to visualize the color style of Hematoxylin and Eosin (H&E) stained WSIs, which is scalable to large real-world cohorts. For this, representative color spectrums are obtained by K-means clustering on slide-level, and the pair-wise distance between spectrums is formulated as a matching problem. Lastly, we use multi-dimensional scaling (MDS) algorithm to obtain 2-dimensional embeddings for WSIs, which are suitable for visualization. We validated the method on lung adenocarcinoma cases and lung squamous-cell carcinoma cases in The Cancer Genome Atlas (TCGA) program. Through the well-visualized staining pattern map, slides with low staining quality or with abnormal staining conditions can be easily recognized. Furthermore, we give a demo usage of the proposed method in the context of a lung cancer segmentation task. Our main conclusions including, (1) biases in staining pattern distribution will harm the performance of CAD systems; (2) weakly stained slides are more challenging than heavily stained slides; (3) stain augmentation can deal with a certain level of staining variation, but not all of it; (4) light stain augmentation can generate more realistic training samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bándi P et al (2017) Comparison of different methods for tissue segmentation in histopathological whole-slide images. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE, pp 591–595

    Chapter  Google Scholar 

  2. Bándi P, Balkenhol M, van Ginneken B, van der Laak J, Litjens G (2019) Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks. PeerJ 7:e8242

    Article  Google Scholar 

  3. Bejnordi BE et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama 318:2199–2210

    Article  Google Scholar 

  4. Clarke EL, Treanor D (2017) Colour in digital pathology: a review. Histopathology 70:153–163

    Article  Google Scholar 

  5. Di Franco C, Bini E, Marinoni M, Buttazzo GC (2017) Multidimensional scaling localization with anchors. In: 2017 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC). IEEE, pp 49–54. https://doi.org/10.1109/ICARSC.2017.7964051

    Chapter  Google Scholar 

  6. Gadermayr M, Gupta L, Appel V, Boor P, Klinkhammer BM, Merhof D (2019) Generative adversarial networks for facilitating stain-independent supervised and unsupervised segmentation: a study on kidney histology. IEEE Trans Med Imaging 38:2293–2302

    Article  Google Scholar 

  7. Gertych A, Swiderska-Chadaj Z, Ma Z, Ing N, Markiewicz T, Cierniak S, Salemi H, Guzman S, Walts AE, Knudsen BS (2019) Convolutional neural networks can accurately distinguish four histologic growth patterns of lung adenocarcinoma in digital slides. Sci Rep 9:1–12

    Google Scholar 

  8. Hanna MG, Reuter VE, Hameed MR, Tan LK, Chiang S, Sigel C, Hollmann T, Giri D, Samboy J, Moradel C, Rosado A, Otilano JR III, England C, Corsale L, Stamelos E, Yagi Y, Schüffler PJ, Fuchs T, Klimstra DS, Sirintrapun SJ (2019) Whole slide imaging equivalency and efficiency study: experience at a large academic center. Mod Pathol 32:916–928

    Article  Google Scholar 

  9. Jiao Y, Li J, Qian C, Fei S (2021) Deep learning-based tumor microenvironment analysis in colon adenocarcinoma histopathological whole-slide images. Computer Methods and Programs in Biomedicine 106047. https://doi.org/10.1016/j.cmpb.2021.106047

  10. Jiao Y, van Rijthoven M, Li J et al (2021)  Automatic Lung Cancer Segmentation in Histopathology Whole-Slide Images with Deep Learning. In: 2021 17th European Congress on Digital Pathology (ECDP2021) 2-2

  11. Khan AM, Rajpoot N, Treanor D, Magee D (2014) A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans Biomed Eng 61:1729–1738

    Article  Google Scholar 

  12. Khan A, Atzori M, Otálora S, Andrearczyk V, Müller H (2020) Generalizing convolution neural networks on stain color heterogeneous data for computational pathology. In: Tomaszewski JE, Ward AD (eds) Medical Imaging 2020: Digital Pathology, vol 26. SPIE. https://doi.org/10.1117/12.2549718

    Chapter  Google Scholar 

  13. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  14. Koledoye MA, Facchinetti T, Almeida L (2017) MDS-based localization with known anchor locations and missing tag-to-tag distances. In: 2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA). IEEE, pp 1–4. https://doi.org/10.1109/ETFA.2017.8247768

    Chapter  Google Scholar 

  15. Kothari S et al (2011) Automatic batch-invariant color segmentation of histological cancer images. In: 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE, pp 657–660. https://doi.org/10.1109/ISBI.2011.5872492

    Chapter  Google Scholar 

  16. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst 25:1097–1105

    Google Scholar 

  17. Lafarge MW, Pluim JPW, Eppenhof KAJ, Veta M (2019) Learning domain-invariant representations of histological images. Front Med 6:162

    Article  Google Scholar 

  18. Li X, Plataniotis KN (2015) A complete color normalization approach to histopathology images using color cues computed from saturation-weighted statistics. IEEE Trans Biomed Eng 62:1862–1873

    Article  Google Scholar 

  19. Li B, Keikhosravi A, Loeffler AG, Eliceiri KW (2021) Single image super-resolution for whole slide image using convolutional neural networks and self-supervised color normalization. Med Image Anal 68:101938

    Article  Google Scholar 

  20. Litjens G et al (2018) 1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset. GigaScience 7:giy065

    Article  Google Scholar 

  21. Macenko M et al (2009) A method for normalizing histology slides for quantitative analysis. In: 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE, pp 1107–1110. https://doi.org/10.1109/ISBI.2009.5193250

    Chapter  Google Scholar 

  22. Magee D et al (2009) Colour Normalisation in Digital Histopathology Images. 12

  23. McInnes L, Healy J, Melville J (2020) UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv:1802.03426 [cs, stat]

  24. Mukhopadhyay S et al (2018) Whole slide imaging versus microscopy for primary diagnosis in surgical pathology. Am J Surg Pathol 42:14

    Article  Google Scholar 

  25. Niethammer M, Borland D, Marron JS, Woosley J, Thomas NE (2010) Appearance Normalization of Histology Slides. In: Wang F, Yan P, Suzuki K, Shen D (eds) Machine Learning in Medical Imaging, vol 6357. Springer, Berlin Heidelberg, pp 58–66

    Chapter  Google Scholar 

  26. Reinhard E, Adhikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Grap Appl 21:34–41

    Article  Google Scholar 

  27. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241

    Google Scholar 

  28. Roy S, Kumar Jain A, Lal S, Kini J (2018) A study about color normalization methods for histopathology images. Micron 114:42–61

    Article  Google Scholar 

  29. Ruifrok AC, Johnston DA (2001) & others. Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol 23:291–299

    Google Scholar 

  30. Salvi M, Michielli N, Molinari F (2020) Stain color adaptive normalization (SCAN) algorithm: separation and standardization of histological stains in digital pathology. Comput Methods Prog Biomed 193:105506

    Article  Google Scholar 

  31. Salvi M, Acharya UR, Molinari F, Meiburger KM (2021) The impact of pre- and post-image processing techniques on deep learning frameworks: a comprehensive review for digital pathology image analysis. Comput Biol Med 128:104129

    Article  Google Scholar 

  32. Senaras C, Niazi MKK, Lozanski G, Gurcan MN (2018) DeepFocus: detection of out-of-focus regions in whole slide digital images using deep learning. PLoS One 13:e0205387

    Article  Google Scholar 

  33. Shin SJ (2021) Style transfer strategy for developing a generalizable deep learning application in digital pathology. Computer Methods and Programs in Biomedicine 9

  34. Shrestha P, Hulsken B (2014) Color accuracy and reproducibility in whole slide imaging scanners. J Med Imag 1:027501

    Article  Google Scholar 

  35. Stacke K, Eilertsen G, Unger J, Lundström C (2019) A closer look at domain shift for deep learning in histopathology. arXiv:1909.11575 [cs]

  36. Stacke K, Eilertsen G, Unger J, Lundstrom C (2021) Measuring domain shift for deep learning in histopathology. IEEE J Biomed Health Inform 25:325–336

    Article  Google Scholar 

  37. Swiderska-Chadaj Z, de Bel T, Blanchet L, Baidoshvili A, Vossen D, van der Laak J, Litjens G (2020) Impact of rescanning and normalization on convolutional neural network performance in multi-center, whole-slide classification of prostate cancer. Sci Rep 10:14398

    Article  Google Scholar 

  38. Tellez D et al (2018) H and E stain augmentation improves generalization of convolutional networks for histopathological mitosis detection. In: Medical Imaging 2018: Digital Pathology vol. 10581 105810Z. International Society for Optics and Photonics

    Google Scholar 

  39. Tellez D, Litjens G, Bándi P, Bulten W, Bokhorst JM, Ciompi F, van der Laak J (2019) Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med Image Anal 58:101544

    Article  Google Scholar 

  40. Tolkach Y, Dohmgörgen T, Toma M, Kristiansen G (2020) High-accuracy prostate cancer pathology using deep learning. Nat Mach Intell 2:411–418

    Article  Google Scholar 

  41. van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  42. Yagi Y (2011) Color standardization and optimization in whole slide imaging. Diagn Pathol 6:S15

    Article  Google Scholar 

  43. Yang M, Nurzynska K, Walts AE, Gertych A (2020) A CNN-based active learning framework to identify mycobacteria in digitized Ziehl-Neelsen stained human tissues. Comput Med Imaging Graph 84:101752

    Article  Google Scholar 

  44. Zarella MD, Yeoh C, Breen DE, Garcia FU (2017) An alternative reference space for H&E color normalization. PLoS One 12:e0174489

    Article  Google Scholar 

  45. Zhu J-Y, Park T, Isola P, Efros A A (2020) Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. arXiv:1703.10593 [cs]

Download references

Availability of data and material

Not applicable.

Code availability

We will make the code publicly available at https://github.com/jiaoyiping630/Stain_visualization.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, China

    Yiping Jiao

  2. School of Automation, Southeast University, Nanjing, China

    Yiping Jiao & Shumin Fei

  3. Luoyang Central Hospital affiliated to Zhengzhou University, Luoyang, China

    Junhong Li

Authors
  1. Yiping Jiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junhong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shumin Fei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shumin Fei.

Ethics declarations

Ethics approval

The whole slide images are publicly available through the TCGA program (https://portal.gdc.cancer.gov/) without any restrictions.

Consent to participate

Not applicable.

Consent for publication

All the authors agree with the submission of this manuscript.

Conflict of interest

The authors declare that no conflicts of interest exist in this manuscript.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiao, Y., Li, J. & Fei, S. Staining condition visualization in digital histopathological whole-slide images. Multimed Tools Appl 81, 17831–17847 (2022). https://doi.org/10.1007/s11042-022-12559-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12559-y

Keywords

Search

Navigation