Skip to main content
SpringerLink
Log in

Deep Learning for Histopathological Image Analysis: Towards Computerized Diagnosis on Cancers

  • Chapter
  • First Online:
Deep Learning and Convolutional Neural Networks for Medical Image Computing

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Automated detection and segmentation of histologic primitives are critical steps for developing computer-aided diagnosis and prognosis system on histopathological tissue specimens. For a number of cancers, the clinical cancer grading system is highly correlated with the pathomic features of histologic primitives that appreciated from histopathological images. However, automated detection and segmentation of histologic primitives is pretty challenged because of the complicity and high density of histologic data. Therefore, there is a high demand for developing intelligent and computational image analysis tools for digital pathology images. Recently there have been interests in the application of “Deep Learning” strategies for classification and analysis of big image data. Histopathology, given its size and complexity, represents an excellent use case for application of deep learning strategies. In this chapter, we present deep learning based approaches for two challenged tasks in histological image analysis: (1) Automated nuclear atypia scoring (NAS) on breast histopathology. We present a Multi-Resolution Convolutional Network (MR-CN) with Plurality Voting (MR-CN-PV) model for automated NAS. MR-CN-PV consists of three Single-Resolution Convolutional Network (SR-CN) with Majority Voting (SR-CN-MV) model for getting independent NAS. MR-CN-PV combines three scores via plurality voting for getting final score. (2) Epithelial (EP) and stromal (ST) tissues discrimination. The work utilized a pixel-wise Convolutional Network (CN-PI) based segmentation model for automated EP and ST tissues discrimination. We present experiments on two challenged datasets. For automated NAS, the MR-CN-PV model was evaluated on MITOS-ATYPIA-14 Challenge dataset. MR-CN-PV model got 67 score which was placed the second comparing with the scores of other five teams. The proposed CN-PI model outperformed patch-wise CN (CN-PA) models in discriminating EP and ST tissues on a breast histological images.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Siegel RL, Miller KD, Jemal A (2015) Cancer statistics, 2015. CA Cancer J Clin 65(1):5–29

    Article  Google Scholar 

  2. Chen W, Zheng R, Baade PD, Zhang S, Zeng H, Bray F, Jemal A, Yu XQ, He J (2016) Cancer statistics in china, 2015. CA: A Cancer J Clin 66(2):115–132

    Google Scholar 

  3. Rorke LB (1997) Pathologic diagnosis as the gold standard. Cancer 79(4):665–667

    Article  Google Scholar 

  4. Rakha E, Reis-Filho J, Baehner F, Dabbs D, Decker T, Eusebi V, Fox S, Ichihara S, Jacquemier J, Lakhani S, Palacios J, Richardson A, Schnitt S, Schmitt F, Tan PH, Tse G, Badve S, Ellis I (2010) Breast cancer prognostic classification in the molecular era: the role of histological grade. Breast Cancer Res 12(4):207

    Article  Google Scholar 

  5. Madabhushi A (2009) Digital pathology image analysis: opportunities and challenges. Imaging Med 1(1):7–10

    Article  Google Scholar 

  6. De Wever O, Mareel M (2003) Role of tissue stroma in cancer cell invasion. J Pathol 200(4):429–447

    Article  Google Scholar 

  7. Downey CL, Simpkins SA, White J, Holliday DL, Jones JL, Jordan LB, Kulka J, Pollock S, Rajan SS, Thygesen HH, Hanby AM, Speirs V (2014) The prognostic significance of tumour-stroma ratio in oestrogen receptor-positive breast cancer. Br J Cancer 110(7):1744–1747

    Article  Google Scholar 

  8. Yuan Y et al (2012) Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling. Sci Transl Med 4(157):157ra143

    Article  Google Scholar 

  9. Bourzac K (2013) Software: the computer will see you now. Nature 502(7473):S92–S94

    Article  Google Scholar 

  10. Meyer JS, Alvarez C, Milikowski C, Olson N, Russo I, Russo J, Glass A, Zehnbauer BA, Lister K, Parwaresch R, Cooperative Breast Cancer Tissue Resource (2005) Breast carcinoma malignancy grading by bloom-richardson system vs proliferation index: reproducibility of grade and advantages of proliferation index. Mod Pathol 18(8):1067–1078

    Google Scholar 

  11. Robbins P, Pinder S, de Klerk N, Dawkins H, Harvey J, Sterrett G, Ellis I, Elston C (1995) Histological grading of breast carcinomas: a study of interobserver agreement. Hum Pathol 26(8):873–879

    Article  Google Scholar 

  12. Brachtel E, Yagi Y (2012) Digital imaging in pathology-current applications and challenges. J Biophotonics 5(4):327–335

    Article  Google Scholar 

  13. Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, Yener B (2009) Histopathological image analysis: a review. IEEE Rev Biomed Eng 2:147–171

    Article  Google Scholar 

  14. Hamilton PW, Bankhead P, Wang YH, Hutchinson R, Kieran D, McArt DG, James J, Salto-Tellez M (2014) Digital pathology and image analysis in tissue biomarker research. Methods 70(1):59–73

    Article  Google Scholar 

  15. Rimm DL (2011) C-path: a watson-like visit to the pathology lab. Sci Transl Med 3(108):108fs8

    Article  Google Scholar 

  16. Anthimopoulos M, Christodoulidis S, Ebner L, Christe A, Mougiakakou S (2016) Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Trans Med Imaging 35(5):1207–1216

    Article  Google Scholar 

  17. Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298

    Article  Google Scholar 

  18. Albarqouni S, Baur C, Achilles F, Belagiannis V, Demirci S, Navab N (2016) AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans Med Imaging 35(5):1313–1321

    Article  Google Scholar 

  19. Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251

    Article  Google Scholar 

  20. Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312

    Article  Google Scholar 

  21. Moeskops P, Viergever MA, Mendrik AM, de Vries LS, Benders MJNL, Isgum I (2016) Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans Med Imaging 35(5):1252–1261

    Article  Google Scholar 

  22. Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel SJ, Wille MMW, Naqibullah M, Snchez CI, van Ginneken B (2016) Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging 35(5):1160–1169

    Article  Google Scholar 

  23. van Grinsven MJJP, van Ginneken B, Hoyng CB, Theelen T, Snchez CI (2016) Fast convolutional neural network training using selective data sampling: application to hemorrhage detection in color fundus images. IEEE Trans Med Imaging 35(5):1273–1284

    Article  Google Scholar 

  24. Liskowski P, Krawiec K (2016) Segmenting retinal blood vessels with deep neural networks. IEEE Trans Med Imaging PP(99):1–1

    Google Scholar 

  25. Ciresan DC et al (2013) Mitosis detection in breast cancer histology images with deep neural networks. In: MICCAI 2013. LNCS, vol 8150. Springer, Berlin, pp 411–418

    Google Scholar 

  26. Xu J, Xiang L, Liu Q, Gilmore H, Wu J, Tang J, Madabhushi A (2016) Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans Med Imaging 35(1):119–130

    Article  Google Scholar 

  27. Sirinukunwattana K, Raza SEA, Tsang YW, Snead D, Cree IA, Rajpoot NM (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35:1196

    Google Scholar 

  28. Xing F, Xie Y, Yang L (2016) An automatic learning-based framework for robust nucleus segmentation. IEEE Trans Med Imaging 35(2):550–566

    Article  Google Scholar 

  29. Xu J, Xiang L, Hang R, Wu J (2014) Stacked sparse autoencoder (SSAE) based framework for nuclei patch classification on breast cancer histopathology. In: ISBI

    Google Scholar 

  30. Xu J, Luo X, Wang G, Gilmore H, Madabhushi A (2016) A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images. Neurocomputing 191:214–223

    Article  Google Scholar 

  31. Cruz-Roa A et al (2013) A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection. In: MICCAI 2013, vol 8150. Springer, Berlin, pp 403–410

    Google Scholar 

  32. Cosatto E, Miller M, Graf HP, Meyer JS (2008) Grading nuclear pleomorphism on histological micrographs. In: 19th International conference on pattern recognition, ICPR 2008, pp 1–4

    Google Scholar 

  33. Lu C, Ji M, Ma Z, Mandal M (2015) Automated image analysis of nuclear atypia in high-power field histopathological image. J Microsc 258(3):233–240

    Article  Google Scholar 

  34. Dalle JR, Li H, Huang CH, Leow WK, Racoceanu D, Putti TC (2009) Nuclear pleomorphism scoring by selective cell nuclei detection

    Google Scholar 

  35. Khan AM, Sirinukunwattana K, Rajpoot N (2015) A global covariance descriptor for nuclear atypia scoring in breast histopathology images. IEEE J Biomed Health Inform 19(5):1637–1647

    Article  Google Scholar 

  36. Shi J, Wu J, Li Y, Zhang Q, Ying S (2016) Histopathological image classification with color pattern random binary hashing based PCANet and matrix-form classifier. IEEE J Biomed Health Inform PP(99):1–1

    Article  Google Scholar 

  37. Basavanhally A, Feldman MD, Shih N, Mies C, Tomaszewski J, Ganesan S, Madabhushi A (2011) Multi-field-of-view strategy for image-based outcome prediction of multi-parametric estrogen receptor-positive breast cancer histopathology: comparison to oncotype DX. J Pathol Inform 2 (01/2012 2011)

    Google Scholar 

  38. Basavanhally A, Ganesan S, Feldman M, Shih N, Mies C, Tomaszewski J, Madabhushi A (2013) Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides. IEEE Trans Biomed Eng 60(8):2089–99

    Google Scholar 

  39. Linder N et al (2012) Identification of tumor epithelium and stroma in tissue microarrays using texture analysis. Diagn Pathol 7(1):22

    Article  Google Scholar 

  40. Bianconi F, lvarez Larrn A, Fernndez A (2015) Discrimination between tumour epithelium and stroma via perception-based features. Neurocomputing 154(0):119–126

    Article  Google Scholar 

  41. Hiary H et al (2013) Automated segmentation of stromal tissue in histology images using a voting bayesian model. Signal Image Video Process 7(6):1229–1237

    Article  Google Scholar 

  42. Eramian M et al (2011) Segmentation of epithelium in H&E stained odontogenic cysts. J Microsc 244(3):273–292

    Article  Google Scholar 

  43. Signolle N, Revenu M, Plancoulaine B, Herlin P (2010) Wavelet-based multiscale texture segmentation: application to stromal compartment characterization on virtual slides. Signal Process 90(8):2412–2422 Special Section on Processing and Analysis of High-Dimensional Masses of Image and Signal Data

    Article  MATH  Google Scholar 

  44. Lahrmann B, Halama N, Sinn HP, Schirmacher P, Jaeger D, Grabe N (2011) Automatic tumor-stroma separation in fluorescence tmas enables the quantitative high-throughput analysis of multiple cancer biomarkers. PLoS ONE 6(12):e28048

    Article  Google Scholar 

  45. Amaral T, McKenna S, Robertson K, Thompson A (2013) Classification and immunohistochemical scoring of breast tissue microarray spots. IEEE Trans Biomed Eng 60(10):2806–2814

    Article  Google Scholar 

  46. Kather JN, Weis CA, Bianconi F, Melchers SM, Schad LR, Gaiser T, Marx A, Zllner FG (2016) Multi-class texture analysis in colorectal cancer histology. Sci Rep 6:27988

    Article  Google Scholar 

  47. Bychkov D, Turkki R, Haglund C, Linder N, Lundin J (2016) Deep learning for tissue microarray image-based outcome prediction in patients with colorectal cancer, vol 9791, pp 979115–979115–6

    Google Scholar 

  48. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

    Google Scholar 

  49. Janowczyk A, Madabhushi A (2016) Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J Pathol Inform 7(1):29

    Article  Google Scholar 

  50. Zhou ZH (2016) Machine learning. Tsinghua University Press, Beijing

    Google Scholar 

  51. ICPR2014 (2010) MITOS & ATYPIA 14 contest. https://grand-challenge.org/site/mitos-atypia-14/dataset/. Accessed 30 Sept 2010

  52. Beck AH et al (2011) Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci Transl Med 3(108):108ra113

    Article  Google Scholar 

  53. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM international conference on multimedia. ACM, pp 675–678

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Nos. 61273259,61272223); Six Major Talents Summit of Jiangsu Province (No. 2013-XXRJ-019), the Natural Science Foundation of Jiangsu Province of China (No. BK20141482), and Jiangsu Innovation & Entrepreneurship Group Talents Plan (No.JS201526).

Author information

Authors and Affiliations

  1. Jiangsu Key Laboratory of Big Data Analysis Technique, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Jun Xu, Chao Zhou, Bing Lang & Qingshan Liu

Authors
  1. Jun Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingshan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Xu .

Editor information

Editors and Affiliations

  1. NIH Clinical Center, Bethesda, Maryland, USA

    Le Lu

  2. Siemens Healthcare Technology Center, Princeton, New Jersey, USA

    Yefeng Zheng

  3. University of Adelaide, Adelaide, South Australia, Australia

    Gustavo Carneiro

  4. University of Florida, Gainesville, Florida, USA

    Lin Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Xu, J., Zhou, C., Lang, B., Liu, Q. (2017). Deep Learning for Histopathological Image Analysis: Towards Computerized Diagnosis on Cancers. In: Lu, L., Zheng, Y., Carneiro, G., Yang, L. (eds) Deep Learning and Convolutional Neural Networks for Medical Image Computing. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-42999-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-42999-1_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-42998-4

  • Online ISBN: 978-3-319-42999-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation