Skip to main content
SpringerLink
Log in

An integrated approach for medical abnormality detection using deep patch convolutional neural networks

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Computer-aided detection of abnormalities in medical images has clinical significance but remains a challenging research topic. Unlike object detection in natural images, the detection of medical abnormalities is unique because they often locate in tiny local regions within a high-resolution medical image. Traditional machine learning approaches build sliding window-based detectors with manual features and are therefore limited on both efficiency and accuracy. Recent advances in deep learning shed new light on this problem, but applying it to medical image analysis faces challenges including insufficient data. Training deep convolutional neural networks (CNNs) directly on high-resolution images requires image compression at the input layer, which leads to the loss of information that is essential for medical abnormality detection. Therefore, instead of training on full images, the proposed approach first fine-tunes the pre-trained deep CNNs on image patches centered at medical abnormalities and then integrates them with class activation mappings and region proposal networks for building abnormality detectors. The deep patch classifier has been tested on a mammogram data set and achieved an overall classification accuracy of \(92.53\%\), compared to \(81.55\%\) by a traditional approach using manual features. The integrated detector has been tested on an ultrasound liver image data set for abnormality detection and achieved an average precision of 0.60, outperforming both the sliding window-based approach of 0.16 and a deep learning YOLO model of 0.51. These validations suggest that the integrated approach has great potential for assisting doctors in detecting abnormalities from medical images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Hernández, P.L.A., Estrada, T.T., Pizarro, A.L., Cisternas, M.L.D., Tapia, C.S.: Calcificaciones mamarias: descripción y clasificación según la 5.a edición bi-rads. Rev. Chilena de Radiol. 22(2), 80–91 (2016). https://doi.org/10.1016/j.rchira.2016.06.004

    Article  Google Scholar 

  2. Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 2843–2851. Curran Associates Inc., New York (2012)

    Google Scholar 

  3. Kooi, T., Litjens, G., van Ginneken, B., Gubern-Mérida, A., Sánchez, C.I., Mann, R., den Heeten, A., Karssemeijer, N.: Large scale deep learning for computer aided detection of mammographic lesions. Med. Image Anal. 35, 303–312 (2017). https://doi.org/10.1016/j.media.2016.07.007

    Article  Google Scholar 

  4. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  5. Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Proceedings of the 27th International Conference on Neural Information Processing Systems—Volume 2, NIPS’14, pp. 3320–3328. MIT Press, Cambridge, MA, USA (2014)

  6. Zhou, B., Khosla, A., Oliva, A.L., Torralba, A.: Learning deep features for discriminative localization. In: CVPR (2016)

  7. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  8. Xi, P., Shu, C., Goubran, R.: Abnormality detection in mammography using deep convolutional neural networks. In: 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA), pp. 1-6 (2018)

  9. Lee, R.S., Gimenez, F., Hoogi, A., Miyake, K.K., Gorovoy, M., Rubin, D.L.: A curated mammography data set for use in computer-aided detection and diagnosis research. Sci. Data 4, 170177 (2017)

    Article  Google Scholar 

  10. Nakayama, R., Uchiyama, Y., Yamamoto, K., Watanabe, R., Namba, K.: Computer-aided diagnosis scheme using a filter bank for detection of microcalcification clusters in mammograms. IEEE Trans. Biomed. Eng. 53(2), 273–283 (2006). https://doi.org/10.1109/TBME.2005.862536

    Article  Google Scholar 

  11. Kavitha, K., Kumaravel, N.: A comparitive study of various microcalcification cluster detection methods in digitized mammograms. In: 2007 14th International Workshop on Systems, Signals and Image Processing and 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services, pp. 405–409 (2007). https://doi.org/10.1109/IWSSIP.2007.4381127

  12. Pal, N.R., Bhowmick, B., Patel, S.K., Pal, S., Das, J.: A multi-stage neural network aided system for detection of microcalcifications in digitized mammograms. Neurocomputing 71(13), 2625–2634 (2008). https://doi.org/10.1016/j.neucom.2007.06.015. Artificial Neural Networks (ICANN 2006) / Engineering of Intelligent Systems (ICEIS 2006)

    Article  Google Scholar 

  13. Harirchi, F., Radparvar, P., Moghaddam, H.A., Dehghan, F., Giti, M.: Two-level algorithm for mcs detection in mammograms using diverse-adaboost-svm. In: 2010 20th International Conference on Pattern Recognition, pp. 269–272 (2010). https://doi.org/10.1109/ICPR.2010.75

  14. Oliver, A., Torrent, A., Lladó, X., Tortajada, M., Tortajada, L., Sentís, M., Freixenet, J., Zwiggelaar, R.: Automatic microcalcification and cluster detection for digital and digitised mammograms. Knowl.-Based Syst. 28, 68–75 (2012). https://doi.org/10.1016/j.knosys.2011.11.021

    Article  Google Scholar 

  15. Zhang, X., Gao, X.: Twin support vector machines and subspace learning methods for microcalcification clusters detection. Eng. Appl. Artif. Intell. 25(5), 1062–1072 (2012). https://doi.org/10.1016/j.engappai.2012.04.003

    Article  Google Scholar 

  16. Li, Y., Chen, H., Cao, L., Ma, J.: A survey of computer-aided detection of breast cancer with mammography. J. Health Med. Inf. (2016). https://doi.org/10.4172/2157-7420.1000238

    Article  Google Scholar 

  17. Petrosian, A., Chan, H.P., Helvie, M.A., Goodsitt, M.M., Adler, D.D.: Computer-aided diagnosis in mammography: classification of mass and normal tissue by texture analysis. Phys. Med. Biol. 39(12), 2273–2288 (1994)

    Article  Google Scholar 

  18. Petrick, N., Chan, H.P., Sahiner, B., Wei, D.: An adaptive density-weighted contrast enhancement filter for mammographic breast mass detection. IEEE Trans. Med. Imaging 15(1), 59–67 (1996). https://doi.org/10.1109/42.481441

    Article  Google Scholar 

  19. Cascio, D., Fauci, F., Magro, R., Raso, G., Bellotti, R., Carlo, F.D., Tangaro, S., Nunzio, G.D., Quarta, M., Forni, G., Lauria, A., Fantacci, M.E., Retico, A., Masala, G.L., Oliva, P., Bagnasco, S., Cheran, S.C., Torres, E.L.: Mammogram segmentation by contour searching and mass lesions classification with neural network. IEEE Trans. Nucl. Sci. 53(5), 2827–2833 (2006). https://doi.org/10.1109/TNS.2006.878003

    Article  Google Scholar 

  20. Han, S., Kang, H.K., Jeong, J.Y., Park, M.H., Kim, W., Bang, W.C., Seong, Y.K.: A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys. Med. Biol. 62(19), 7714 (2017)

    Article  Google Scholar 

  21. Yap, M.H., Pons, G., Martí, J., Ganau, S., Sentís, M., Zwiggelaar, R., Davison, A.K., Martí, R.: Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J. Biomed. Health Inform. 22(4), 1218–1226 (2018). https://doi.org/10.1109/JBHI.2017.2731873

    Article  Google Scholar 

  22. Li, H., Weng, J., Shi, Y., Gu, W., Mao, Y., Wang, Y., Liu, W., Zhang, J.: An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images. Sci. Rep. 8(1), 6600 (2018). https://doi.org/10.1038/s41598-018-25005-7

    Article  Google Scholar 

  23. Chi, J., Walia, E., Babyn, P., Wang, J., Groot, G., Eramian, M.: Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J. Digit. Imaging 30(4), 477–486 (2017). https://doi.org/10.1007/s10278-017-9997-y

    Article  Google Scholar 

  24. Mazurowski, M.A., Buda, M., Saha, A., Bashir, M.R.: Deep learning in radiology: an overview of the concepts and a survey of the state of the art. CoRR arXiv:1802.08717 (2018)

  25. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. CoRR arXiv:1705.02315 (2017)

  26. Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C., Shpanskaya, K., Lungren, M.P., Ng, A.Y.: Chexnet: radiologist-level pneumonia detection on chest x-rays with deep learning. CoRR arXiv:1711.05225 (2017)

  27. Rajpurkar, P., Irvin, J., Bagul, A., Ding, D., Duan, T., Mehta, H., Yang, B., Zhu, K., Laird, D., Ball, R.L., Langlotz, C., Shpanskaya, K., Lungren, M.P., Ng, A.: Mura dataset: towards radiologist-level abnormality detection in musculoskeletal radiographs. arXiv arXiv:1712.06957v3 (2017)

  28. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates Inc., New York (2012)

    Google Scholar 

  29. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR arXiv:1409.1556 (2014)

  30. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition (CVPR) (2015). arXiv:1409.4842

  31. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR arXiv:1512.03385 (2015)

  32. Xi, P., Goubran, R., Shu, C.: Cardiac murmur classification in phonocardiograms using deep convolutional neural networks. In: Proceedings of the International Conference on Pattern Recognition and Artificial Intelligence (2018)

  33. Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010). https://doi.org/10.1109/TPAMI.2009.167

    Article  Google Scholar 

  34. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’14, pp. 580–587. IEEE Computer Society, Washington, DC, USA (2014). https://doi.org/10.1109/CVPR.2014.81

  35. Sa, R., Owens, W., Wiegand, R., Chaudhary, V.: Fast scale-invariant lateral lumbar vertebrae detection and segmentation in x-ray images. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1054–1057 (2016). https://doi.org/10.1109/EMBC.2016.7590884

  36. Girshick, R.B.: Fast R-CNN. CoRR arXiv:1504.08083 (2015)

  37. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision—ECCV 2016, pp. 21–37. Springer, Cham (2016)

    Chapter  Google Scholar 

  38. Dai, J., Li, Y., He, K., Sun, J.: R-fcn: object detection via region-based fully convolutional networks. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29, pp. 379–387. Curran Associates Inc., New York (2016)

    Google Scholar 

  39. Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., Murphy, K.: Speed/accuracy trade-offs for modern convolutional object detectors. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3296–3297 (2017). https://doi.org/10.1109/CVPR.2017.351

  40. Suckling, J.: The mammographic image analysis society digital mammogram database. Exerpta Med. Int. Congr. Ser. 1069, 375–378 (1994)

    Google Scholar 

  41. Heath, M., Bowyer, K., Kopans, D., Moore, R., Kegelmeyer, W.P.: The digital database for screening mammography. In: Proceedings of the Fifth International Workshop on Digital Mammography, pp. 212–218 (2001)

  42. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 886–893 (2005). https://doi.org/10.1109/CVPR.2005.177

  43. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

Download references

Author information

Authors and Affiliations

  1. Carleton University, 1125 Colonel By Dr., Ottawa, ON, K1S 5B6, Canada

    Pengcheng Xi & Rafik Goubran

  2. National Research Council Canada, 1200 Montreal Rd., Ottawa, Ontario, K1A0R6, Canada

    Pengcheng Xi, Chang Shu & Louis Borgeat

  3. Nantong No. 3 People’s Hospital, 99 Qingnian Middle Rd., Nantong, 226006, Jiangsu, China

    Haitao Guan

Authors
  1. Pengcheng Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haitao Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chang Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Louis Borgeat
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rafik Goubran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Pengcheng Xi or Haitao Guan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xi, P., Guan, H., Shu, C. et al. An integrated approach for medical abnormality detection using deep patch convolutional neural networks. Vis Comput 36, 1869–1882 (2020). https://doi.org/10.1007/s00371-019-01775-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01775-7

Keywords

Search

Navigation