Skip to main content
Springer Nature Link
Log in

Three Aspects on Using Convolutional Neural Networks for Computer-Aided Detection in Medical Imaging

  • Chapter
  • First Online:
Deep Learning and Convolutional Neural Networks for Medical Image Computing

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

  • 9267 Accesses

Abstract

Deep convolutional neural networks (CNNs) enable learning trainable, highly representative and hierarchical image feature from sufficient training data which makes rapid progress in computer vision possible. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pretrained CNN features, and transfer learning , i.e., fine-tuning CNN models pretrained from natural image dataset (such as large-scale annotated natural image database: ImageNet) to medical image tasks. In this chapter, we exploit three important factors of employing deep convolutional neural networks to computer-aided detection problems. First, we exploit and evaluate several different CNN architectures including from shallower to deeper CNNs: classical CifarNet, to recent AlexNet and state-of-the-art GoogLeNet and their variants. The studied models contain five thousand to 160 million parameters and vary in the numbers of layers. Second, we explore the influence of dataset scales and spatial image context configurations on medical image classification performance. Third, when and why transfer learning from the pretrained ImageNet CNN models (via fine-tuning) can be useful for medical imaging tasks are carefully examined. We study two specific computer-aided detection (CADe) problems, namely thoracoabdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection and report the first fivefold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive quantitative evaluation, CNN model analysis, and empirical insights can be helpful to the design of high-performance CAD systems for other medical imaging tasks, without loss of generality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This can be achieved by segmenting the lung using simple label fusion methods [46]. First, we overlay the target image slice with the average lung mask among the training folds. Second, we perform simple morphology operations to obtain the lung boundary.

References

  1. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: IEEE CVPR

    Google Scholar 

  2. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg A, Fei-Fei L (2014) Imagenet large scale visual recognition challenge. arXiv:1409.0575

  3. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  4. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: NIPS, pp 1097–1105

    Google Scholar 

  5. Krizhevsky A (2009) Learning multiple layers of features from tiny images, in Master’s Thesis. University of Toronto, Department of Computer Science

    Google Scholar 

  6. Girshick R, Donahue J, Darrell T, Malik J (2015) Region-based convolutional networks for accurate object detection and semantic segmentation. In: IEEE Transaction Pattern Analysis Machine Intelligence

    Google Scholar 

  7. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transaction Pattern Analysis Machine Intelligence

    Google Scholar 

  8. Everingham M, Eslami SMA, Van Gool L, Williams C, Winn J, Zisserman A (2015) The pascal visual object classes challenge: a retrospective. Int J Comput Vis 111(1):98–136

    Article  Google Scholar 

  9. van Ginneken B, Setio A, Jacobs C, Ciompi F (2015) Off-the-shelf convolutional neural network features for pulmonary nodule detection in computed tomography scans. In: IEEE ISBI, pp 286–289

    Google Scholar 

  10. Bar Y, Diamant I, Greenspan H, Wolf L (2015) Chest pathology detection using deep learning with non-medical training. In: IEEE ISBI

    Google Scholar 

  11. Shin H, Lu L, Kim L, Seff A, Yao J, Summers R (2015) Interleaved text/image deep mining on a large-scale radiology image database. In: IEEE Conference on CVPR, pp 1–10

    Google Scholar 

  12. Ciompi F, de Hoop B, van Riel SJ, Chung K, Scholten E, Oudkerk M, de Jong P, Prokop M, van Ginneken B (2015) Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2d views and a convolutional neural network out-of-the-box. Med Image Anal 26(1):195–202

    Article  Google Scholar 

  13. Menze B, Reyes M, Van Leemput K (2015) The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans Med Imaging 34(10):1993–2024

    Article  Google Scholar 

  14. Pan Y, Huang W, Lin Z, Zhu W, Zhou J, Wong J, Ding Z (2015) Brain tumor grading based on neural networks and convolutional neural networks. In: IEEE EMBC, pp 699–702

    Google Scholar 

  15. Shen W, Zhou M, Yang F, Yang C, Tian J (2015) Multi-scale convolutional neural networks for lung nodule classification. In: IPMI, pp 588–599

    Google Scholar 

  16. Carneiro G, Nascimento J, Bradley AP (2015) Unregistered multiview mammogram analysis with pre-trained deep learning models. In: MICCAI, pp 652–660

    Google Scholar 

  17. Wolterink JM, Leiner T, Viergever MA, Išgum I (2015) Automatic coronary calcium scoring in cardiac CT angiography using convolutional neural networks. In: MICCAI, pp 589–596

    Google Scholar 

  18. Schlegl T, Ofner J, Langs G (2014) Unsupervised pre-training across image domains improves lung tissue classification. In: Medical computer vision: algorithms for big data. Springer, Berlin, pp 82–93

    Google Scholar 

  19. Hofmanninger J, Langs G (2015) Mapping visual features to semantic profiles for retrieval in medical imaging. In: IEEE conference on CVPR

    Google Scholar 

  20. Carneiro G, Nascimento J (2013) Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data. IEEE Trans Pattern Anal Mach Intell 35(11):2592–2607

    Article  Google Scholar 

  21. Li R, Zhang W, Suk H, Wang L, Li J, Shen D, Ji S (2014) Deep learning based imaging data completion for improved brain disease diagnosis. In: MICCAI

    Google Scholar 

  22. Barbu A, Suehling M, Xu X, Liu D, Zhou SK, Comaniciu D (2012) Automatic detection and segmentation of lymph nodes from CT data. IEEE Trans Med Imaging 31(2):240–250

    Article  Google Scholar 

  23. Feulner J, Zhou SK, Hammon M, Hornegger J, Comaniciu D (2013) Lymph node detection and segmentation in chest CT data using discriminative learning and a spatial prior. Med Image Anal 17(2):254–270

    Article  Google Scholar 

  24. Feuerstein M, Glocker B, Kitasaka T, Nakamura Y, Iwano S, Mori K (2012) Mediastinal atlas creation from 3-d chest computed tomography images: application to automated detection and station mapping of lymph nodes. Med Image Anal 16(1):63–74

    Article  Google Scholar 

  25. Lu L, Devarakota P, Vikal S, Wu D, Zheng Y, Wolf M (2014) Computer aided diagnosis using multilevel image features on large-scale evaluation. In: Medical computer vision. Large data in medical imaging. Springer, Berlin, pp 161–174

    Google Scholar 

  26. Lu L, Bi J, Wolf M, Salganicoff M (2011) Effective 3d object detection and regression using probabilistic segmentation features in CT images. In: IEEE CVPR

    Google Scholar 

  27. Roth H, Lu L, Liu J, Yao J, Seff A, Cherry KM, Turkbey E, Summers R (2016) Improving computer-aided detection using convolutional neural networks and random view aggregation. In: IEEE Transaction on Medical Imaging

    Google Scholar 

  28. Lu L, Barbu A, Wolf M, Liang J, Salganicoff M, Comaniciu D (2008) Accurate polyp segmentation for 3d CT colonography using multi-staged probabilistic binary learning and compositional model. In: IEEE CVPR

    Google Scholar 

  29. Tajbakhsh N, Gotway MB, Liang J (2015) Computer-aided pulmonary embolism detection using a novel vessel-aligned multi-planar image representation and convolutional neural networks. In: MICCAI

    Google Scholar 

  30. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  31. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE CVPR, vol 1, pp 886–893

    Google Scholar 

  32. Torralba A, Fergus R, Weiss Y (2008) Small codes and large image databases for recognition. In: IEEE CVPR, pp 1–8

    Google Scholar 

  33. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Rabinovich A (2015) Going deeper with convolutions. In: IEEE conference on CVPR

    Google Scholar 

  34. Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2015) Return of the devil in the details: delving deep into convolutional nets. In: BMVC

    Google Scholar 

  35. Chatfield K, Lempitsky VS, Vedaldi A, Zisserman A (2011) The devil is in the details: an evaluation of recent feature encoding methods. In: BMVC

    Google Scholar 

  36. Seff A, Lu L, Barbu A, Roth H, Shin H-C, Summers R (2015) Leveraging mid-level semantic boundary cues for computer-aided lymph node detection. In: MICCAI

    Google Scholar 

  37. Depeursinge A, Vargas A, Platon A, Geissbuhler A, Poletti P-A, Müller H (2012) Building a reference multimedia database for interstitial lung diseases. Comput Med Imaging Graph 36(3):227–238

    Article  Google Scholar 

  38. Song Y, Cai W, Zhou Y, Feng DD (2013) Feature-based image patch approximation for lung tissue classification. IEEE Trans Med Imaging 32(4):797–808

    Article  Google Scholar 

  39. Song Y, Cai W, Huang H, Zhou Y, Feng D, Wang Y, Fulham M, Chen M (2015) Large margin local estimate with applications to medical image classification. IEEE Transaction on Medical Imaging

    Google Scholar 

  40. Seff A, Lu L, Cherry KM, Roth HR, Liu J, Wang S, Hoffman J, Turkbey EB, Summers R (2014) 2d view aggregation for lymph node detection using a shallow hierarchy of linear classifiers. In: MICCAI, pp 544–552

    Google Scholar 

  41. Gao M, Bagci U, Lu L, Wu A, Buty M, Shin H-C, Roth H, Papadakis ZG, Depeursinge A, Summers R, Xu Z, Mollura JD (2015) Holistic classification of CT attenuation patterns for interstitial lung diseases via deep convolutional neural networks. In: MICCAI first workshop on deep learning in medical image analysis

    Google Scholar 

  42. Lu L, Liu M, Ye X, Yu S, Huang H (2011) Coarse-to-fine classification via parametric and nonparametric models for computer-aided diagnosis. In: ACM conference on CIKM, pp 2509–2512

    Google Scholar 

  43. Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 35(8):1915–1929

    Article  Google Scholar 

  44. Mostajabi M, Yadollahpour P, Shakhnarovich G (2014) Feedforward semantic segmentation with zoom-out features. arXiv:1412.0774

  45. Gao M, Xu Z, Lu L, Nogues I, Summers R, Mollura D (2016) Segmentation label propagation using deep convolutional neural networks and dense conditional random field. In: IEEE ISBI

    Google Scholar 

  46. Wang H, Suh JW, Das SR, Pluta JB, Craige C, Yushkevich P et al (2013) Multi-atlas segmentation with joint label fusion. IEEE Trans Pattern Anal Mach Intell 35(3):611–623

    Article  Google Scholar 

  47. Oquab M, Bottou L, Laptev I, Sivic J (2015) Is object localization for free?–weakly-supervised learning with convolutional neural networks. In: IEEE CVPR, pp 685–694

    Google Scholar 

  48. Oquab M, Bottou L, Laptev I, Josef S (2015) Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE CVPR, pp 1717–1724

    Google Scholar 

  49. Zhu X, Vondrick C, Ramanan D, Fowlkes C (2012) Do we need more training data or better models for object detection. In: BMVC

    Google Scholar 

  50. Ciresan D, Giusti A, Gambardella L, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. In: MICCAI

    Google Scholar 

  51. Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, Shen D (2015) Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108:214–224

    Article  Google Scholar 

  52. Li Q, Cai W, Wang X, Zhou Y, Feng DD, Chen M (2014) Medical image classification with convolutional neural network. In: IEEE ICARCV, pp 844–848

    Google Scholar 

  53. Miller GA (1995) Wordnet: a lexical database for english. Commun ACM 38(11):39–41

    Article  Google Scholar 

  54. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick RB, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. ACM Multimed 2:4

    Google Scholar 

  55. Razavian AS, Azizpour H, Sullivan J, Carlsson S (2014) Cnn features off-the-shelf: an astounding baseline for recognition. In: IEEE CVPRW, pp. 512–519

    Google Scholar 

  56. Zhou B, Lapedriza A, Xiao J, Torralba A, Oliva A (2014) Learning deep features for scene recognition using places database. In: NIPS, pp 487–495

    Google Scholar 

  57. Gupta S, Girshick R, Arbelez P, Malik J (2014) Learning rich features from rgb-d images for object detection and segmentation. In: ECCV, pp 345–360

    Google Scholar 

  58. Gupta S, Arbelez P, Girshick R, Malik J (2015) Indoor scene understanding with rgb-d images: bottom-up segmentation, object detection and semantic segmentation. Int J Comput Vis 112(2):133–149

    Article  MathSciNet  Google Scholar 

  59. Gupta A, Ayhan M, Maida A (2013) Natural image bases to represent neuroimaging data. In: ICML, pp 987–994

    Google Scholar 

  60. Chen H, Dou Q, Ni D, Cheng J, Qin J, Li S, Heng P (2015) Automatic fetal ultrasound standard plane detection using knowledge transferred recurrent neural networks. In: MICCAI, pp 507–514

    Google Scholar 

  61. Roth H, Lu L, Farag A, Shin H-C, Liu J, Turkbey E, Summers R (2015) Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation. In: MICCAI

    Google Scholar 

  62. Kim L, Roth H, Lu L, Wang S, Turkbey E, Summers R (2014) Performance assessment of retroperitoneal lymph node computer-assisted detection using random forest and deep convolutional neural network learning algorithms in tandem. In: The 102nd annual meeting of radiological society of North America

    Google Scholar 

  63. Holmes III D, Bartholmai B, Karwoski R, Zavaletta V, Robb R (2006) The lung tissue research consortium: an extensive open database containing histological, clinical, and radiological data to study chronic lung disease. In: 2006 MICCAI open science workshop

    Google Scholar 

  64. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y (2014) Overfeat: integrated recognition, localization and detection using convolutional networks. In: ICLR

    Google Scholar 

  65. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. In: ICLR

    Google Scholar 

  66. Hochreiter S (1998) The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int J Uncertain Fuzziness Knowl-Based Syst 6(02):107–116

    Article  MATH  Google Scholar 

  67. Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  68. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166

    Article  Google Scholar 

  69. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: ECCV, pp 818–833

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, MD, 20837, USA

    Hoo-Chang Shin, Holger R. Roth, Mingchen Gao, Le Lu, Ziyue Xu, Isabella Nogues, Jianhua Yao, Daniel Mollura & Ronald M. Summers

Authors
  1. Hoo-Chang Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Holger R. Roth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingchen Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Le Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ziyue Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Isabella Nogues
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jianhua Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Daniel Mollura
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ronald M. Summers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hoo-Chang Shin .

Editor information

Editors and Affiliations

  1. NIH Clinical Center, Bethesda, Maryland, USA

    Le Lu

  2. Siemens Healthcare Technology Center, Princeton, New Jersey, USA

    Yefeng Zheng

  3. University of Adelaide, Adelaide, South Australia, Australia

    Gustavo Carneiro

  4. University of Florida, Gainesville, Florida, USA

    Lin Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Shin, HC. et al. (2017). Three Aspects on Using Convolutional Neural Networks for Computer-Aided Detection in Medical Imaging. In: Lu, L., Zheng, Y., Carneiro, G., Yang, L. (eds) Deep Learning and Convolutional Neural Networks for Medical Image Computing. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-42999-1_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-42999-1_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-42998-4

  • Online ISBN: 978-3-319-42999-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics