Skip to main content
SpringerLink
Log in

Deep Learning and Computer-Aided Diagnosis for Medical Image Processing: A Personal Perspective

  • Chapter
  • First Online:
Deep Learning and Convolutional Neural Networks for Medical Image Computing

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

These are exciting times for medical image processing. Innovations in deep learning and the increasing availability of large annotated medical image datasets are leading to dramatic advances in automated understanding of medical images. From this perspective, I give a personal view of how computer-aided diagnosis of medical images has evolved and how the latest advances are leading to dramatic improvements today. I discuss the impact of deep learning on automated disease detection and organ and lesion segmentation, with particular attention to applications in diagnostic radiology. I provide some examples of how time-intensive and expensive manual annotation of huge medical image datasets by experts can be sidestepped by using weakly supervised learning from routine clinically generated medical reports. Finally, I identify the remaining knowledge gaps that must be overcome to achieve clinician-level performance of automated medical image processing systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Giger ML, Chan HP, Boone J (2008) Anniversary paper: history and status of CAD and quantitative image analysis: the role of Medical Physics and AAPM. Med Phys 35(12):5799–5820

    Article  Google Scholar 

  2. Willems JL, Abreu-Lima C, Arnaud P, van Bemmel JH, Brohet C, Degani R, Denis B, Gehring J, Graham I, van Herpen G et al (1991) The diagnostic performance of computer programs for the interpretation of electrocardiograms. N Engl J Med 325(25):1767–1773

    Article  Google Scholar 

  3. Rubin JM, Sayre RE (1978) 1978 memorial award paper: a computer-aided technique for overlaying cerebral angiograms onto computed tomograms. Invest Radiol 13(5):362–367

    Article  Google Scholar 

  4. Fujita H, Doi K, Fencil LE, Chua KG (1987) Image feature analysis and computer-aided diagnosis in digital radiography. 2. Computerized determination of vessel sizes in digital subtraction angiography. Med Phys 14(4):549–556

    Article  Google Scholar 

  5. Xing F, Yang L (2016) Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: a comprehensive review. IEEE Rev Biomed Eng 9:234–263

    Article  Google Scholar 

  6. Harris G (2010) CT scans cut lung cancer deaths, study finds, The New York Times. http://www.nytimes.com/2010/11/05/health/research/05cancer.html. 4 Nov 2010

  7. Summers RM (2003) Road maps for advancement of radiologic computer-aided detection in the 21st century. Radiology 229(1):11–13

    Article  Google Scholar 

  8. Summers RM, Selbie WS, Malley JD, Pusanik L, Dwyer AJ, Courcoutsakis N, Kleiner DE, Sneller MC, Langford C, Shelhamer JH (1997) Computer-assisted detection of endobronchial lesions using virtual bronchoscopy: application of concepts from differential geometry. In: Conference on mathematical models in medical and health sciences, Vanderbilt University

    Google Scholar 

  9. Summers RM, Pusanik LM, Malley JD (1998) Automatic detection of endobronchial lesions with virtual bronchoscopy: comparison of two methods. Proc SPIE 3338:327–335

    Article  Google Scholar 

  10. Summers RM, Selbie WS, Malley JD, Pusanik LM, Dwyer AJ, Courcoutsakis N, Shaw DJ, Kleiner DE, Sneller MC, Langford CA, Holland SM, Shelhamer JH (1998) Polypoid lesions of airways: early experience with computer-assisted detection by using virtual bronchoscopy and surface curvature. Radiology 208:331–337

    Article  Google Scholar 

  11. Summers RM, Yao J, Pickhardt PJ, Franaszek M, Bitter I, Brickman D, Krishna V, Choi JR (2005) Computed tomographic virtual colonoscopy computer-aided polyp detection in a screening population. Gastroenterology 129(6):1832–1844

    Article  Google Scholar 

  12. Dachman AH, Obuchowski NA, Hoffmeister JW, Hinshaw JL, Frew MI, Winter TC, Van Uitert RL, Periaswamy S, Summers RM, Hillman BJ (2010) Effect of computer-aided detection for CT colonography in a multireader, multicase trial. Radiology 256(3):827–835

    Article  Google Scholar 

  13. Wang S, Summers RM (2012) Machine learning and radiology. Med Image Anal 16(5):933–951

    Article  Google Scholar 

  14. Jerebko AK, Malley JD, Franaszek M, Summers RM (2003) Multiple neural network classification scheme for detection of colonic polyps in CT colonography data sets. Acad Radiol 10(2):154–160

    Article  Google Scholar 

  15. Jerebko AK, Malley JD, Franaszek M, Summers RM (2003) Computer-aided polyp detection in CT colonography using an ensemble of support vector machines. In: Cars 2003: computer assisted radiology and surgery, proceedings, vol 1256, pp 1019–1024

    Google Scholar 

  16. Jerebko AK, Summers RM, Malley JD, Franaszek M, Johnson CD (2003) Computer-assisted detection of colonic polyps with CT colonography using neural networks and binary classification trees. Med Phys 30(1):52–60

    Article  Google Scholar 

  17. Malley JD, Jerebko AK, Summers RM (2003) Committee of support vector machines for detection of colonic polyps from CT scans, pp 570–578

    Google Scholar 

  18. Jerebko AK, Malley JD, Franaszek M, Summers RM (2005) Support vector machines committee classification method for computer-aided polyp detection in CT colonography. Acad Radiol 12(4):479–486

    Article  Google Scholar 

  19. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  20. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database, 248–255

    Google Scholar 

  21. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  22. Sahiner B, Chan HP, Petrick N, Wei DT, Helvie MA, Adler DD, Goodsitt MM (1996) Classification of mass and normal breast tissue: a convolution neural network classifier with spatial domain and texture images. IEEE Trans Med Imaging 15(5):598–610

    Article  Google Scholar 

  23. Chan HP, Lo SC, Sahiner B, Lam KL, Helvie MA (1995) Computer-aided detection of mammographic microcalcifications: pattern recognition with an artificial neural network. Med Phys 22(10):1555–1567

    Article  Google Scholar 

  24. Gulcehre C (2016) Deep Learning Software Links. http://deeplearning.net/software_links/. Accessed 18 May 2016

  25. Shin H-C, Lu L, Kim L, Seff A, Yao J, Summers RM (2015) Interleaved text/image deep mining on a very large-scale radiology database. In: The IEEE conference on computer vision and pattern recognition (CVPR), pp 1090–1099

    Google Scholar 

  26. Shin H.-C, Roberts K, Lu L, Demner-Fushman D, Yao J, Summers RM (2016) Learning to read chest X-rays: recurrent neural cascade model for automated image annotation. arXiv preprint arXiv:1603.08486

  27. Wang X, Lu L, Shin H, Kim L, Bagheri M, Nogues I, Yao J, Summers RM (2017) Unsupervised joint mining of deep features and image labels for large-scale radiology image annotation and scene recognition. IEEE Winter Conference on Applications of Computer Vision (WACV)pp 998–1007. arXiv:1701.06599

  28. Roth HR, Lu L, Liu J, Yao J, Seff A, Cherry K, Kim L, Summers RM (2016) Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE TMI 35(5):1170–1181

    Google Scholar 

  29. Shin H-C, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298

    Article  Google Scholar 

  30. Greenspan H, van Ginneken B, Summers RM (2016) Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging 35(5):1153–1159

    Article  Google Scholar 

  31. Krishna R, Zhu Y, Groth O, Johnson J, Hata K, Kravitz J, Chen S, Kalantidis Y, Li L.-J, Shamma DA (2016) Visual genome: connecting language and vision using crowdsourced dense image annotations. arXiv preprint arXiv:1602.07332

  32. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385

  33. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  34. Jaderberg M, Simonyan K, Zisserman A (2015) Spatial transformer networks. In: Advances in neural information processing systems pp 2008–2016

    Google Scholar 

  35. Ciompi F, de Hoop B, van Riel SJ, Chung K, Scholten ET, Oudkerk M, de Jong PA, Prokop M, van Ginneken B (2015) Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box. Med Image Anal 26(1):195–202

    Article  Google Scholar 

  36. Anavi Y, Kogan I, Gelbart E, Geva O, Greenspan H (2015) A comparative study for chest radiograph image retrieval using binary, texture and deep learning classification. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 2940–2943

    Google Scholar 

  37. Roth HR, Lu L, Farag A, Shin H-C, Liu J, Turkbey EB, Summers RM (2015) DeepOrgan: multi-level deep convolutional networks for automated pancreas segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical image computing and computer-assisted intervention – MICCAI 2015, Part I, vol 9349. LNCS. Springer, Heidelberg, pp 556–564

    Google Scholar 

  38. Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M (2013) Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. Med Image Comput Comput Assist Interv 16(Pt 2):246–253

    Google Scholar 

  39. Grand challenges in biomedical image analysis. http://grand-challenge.org/. Accessed 18 May 2016

  40. VISCERAL. http://www.visceral.eu/. Accessed 14 Dec 2015

  41. Roth HR, Summers RM (2015) CT lymph nodes. https://wiki.cancerimagingarchive.net/display/Public/CT+Lymph+Nodes. Accessed 14 Dec 2015

  42. Roth HR, Farag A, Turkbey E.B, Lu L, Liu J, Summers RM (2016) Data from Pancreas-CT. The cancer imaging archive. https://wiki.cancerimagingarchive.net/display/Public/Pancreas-CT. Accessed 18 May 2016

  43. Nguyen TB, Wang SJ, Anugu V, Rose N, McKenna M, Petrick N, Burns JE, Summers RM (2012) Distributed human intelligence for colonic polyp classification in computer-aided detection for CT colonography. Radiology 262(3):824–833

    Article  Google Scholar 

  44. McKenna MT, Wang S, Nguyen TB, Burns JE, Petrick N, Summers RM (2012) Strategies for improved interpretation of computer-aided detections for CT colonography utilizing distributed human intelligence. Med Image Anal 16(6):1280–1292

    Article  Google Scholar 

  45. Albarqouni S, Baur C, Achilles F, Belagiannis V, Demirci S, Navab N (2016) AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans Med Imaging 35(5):1313–1321

    Article  Google Scholar 

  46. Summers RM (21 April 2016) Progress in fully automated abdominal CT interpretation. AJR Am J Roentgenol, 1–13

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Intramural Research Program of the National Institutes of Health, Clinical Center.

Author information

Authors and Affiliations

  1. Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bldg. 10, Room 1C224D MSC 1182, Bethesda, MD, 20892-1182, USA

    Ronald M. Summers

Authors
  1. Ronald M. Summers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ronald M. Summers .

Editor information

Editors and Affiliations

  1. NIH Clinical Center, Bethesda, Maryland, USA

    Le Lu

  2. Siemens Healthcare Technology Center, Princeton, New Jersey, USA

    Yefeng Zheng

  3. University of Adelaide, Adelaide, South Australia, Australia

    Gustavo Carneiro

  4. University of Florida, Gainesville, Florida, USA

    Lin Yang

Additional information

Financial Disclosure The author receives patent royalties from iCAD Medical.

Disclaimer No NIH endorsement of any product or company mentioned in this manuscript should be inferred. The opinions expressed herein are the author’s and do not necessarily represent those of NIH.

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Summers, R.M. (2017). Deep Learning and Computer-Aided Diagnosis for Medical Image Processing: A Personal Perspective. In: Lu, L., Zheng, Y., Carneiro, G., Yang, L. (eds) Deep Learning and Convolutional Neural Networks for Medical Image Computing. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-42999-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-42999-1_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-42998-4

  • Online ISBN: 978-3-319-42999-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation