Skip to main content
SpringerLink
Log in

Development of training environment for deep learning with medical images on supercomputer system based on asynchronous parallel Bayesian optimization

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Recently, deep learning has been exploited in the field of medical image analysis. However, the training of deep learning models with medical images is time-consuming since most medical image data are three-dimensional volumes or high-resolution two-dimensional images. Moreover, the optimization of numerous hyperparameters strongly affects the performance of deep learning. If a framework for training deep learning with hyperparameter optimization on a supercomputer system can be realized, it is expected to accelerate the training of deep learning with medical images. In this study, we described our novel environment for training deep learning with medical images on the supercomputer system in our institute (Reedbush-H supercomputer system) based on asynchronous parallel Bayesian optimization. We trained two types of automated lesion detection application in a constructed environment. The constructed environment enabled us to train deep learning with hyperparameter tuning in a short time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M (2016) TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv e-prints arXiv:1603.04467

  2. Armato RSG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA, Kazerooni EA, MacMahon H, van Beeke EJ, Yankelevitz D, Biancardi AM, Bland PH, Brown MS, Engelmann RM, Laderach GE, Max D, Pais RC, Qing DP, Roberts RY, Smith AR, Starkey A, Batrah P, Caligiuri P, Farooqi A, Gladish GW, Jude CM, Munden RF, Petkovska I, Quint LE, Schwartz LH, Sundaram B, Dodd LE, Fenimore C, Gur D, Petrick N, Freymann J, Kirby J, Hughes B, Casteele AV, Gupte S, Sallamm M, Heath MD, Kuhn MH, Dharaiya E, Burns R, Fryd DS, Salganicoff M, Anand V, Shreter U, Vastagh S, Croft BY (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on ct scans. Med Phys 38(2):915–31. https://doi.org/10.1118/1.3528204

    Article  Google Scholar 

  3. Balaprakash P, Salim M, Uram TD, Vishwanath V, Wild SM (2018) Deephyper: asynchronous hyperparameter search for deep neural networks. In: 2018 IEEE 25th international conference on high performance computing (HiPC), pp 42–51

  4. Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13(Feb):281–305

    MathSciNet  MATH  Google Scholar 

  5. Chakraborty DP, Berbaum KS (2004) Observer studies involving detection and localization: modeling, analysis, and validation. Med Phys 31(8):2313–30. https://doi.org/10.1118/1.1769352

    Article  Google Scholar 

  6. Chollet F (2015) Keras. https://github.com/fchollet/keras. Accessed 19 Jan 2020

  7. Contal E, Perchet V, Vayatis N (2014) Gaussian process optimization with mutual information. In: Proceedings of the 31st international conference on machine learning, vol 32, pp 253–261

  8. Gerard P, Kapadia N, Chang PT, Acharya J, Seiler M, Lefkovitz Z (2013) Extended outlook: description, utilization, and daily applications of cloud technology in radiology. AJR Am J Roentgenol 201(6):W809–11. https://doi.org/10.2214/ajr.12.9673

    Article  Google Scholar 

  9. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv e-prints arXiv:1512.03385

  10. Hiraishi T, Abe T, Iwashita T, Nakashima H (2012) Xcrypt: a perl extension for job level parallel programming. In: Proceedings of the WHIST 2012

  11. Kagadis GC, Kloukinas C, Moore K, Philbin J, Papadimitroulas P, Alexakos C, Nagy PG, Visvikis D, Hendee WR (2013) Cloud computing in medical imaging. Med Phys 40(7):070901. https://doi.org/10.1118/1.4811272

    Article  Google Scholar 

  12. Kandasamy K, Krishnamurthy A, Schneider J, Poczos B (2018) Parallelised Bayesian optimisation via Thompson sampling. In: Proceedings of the 21st international conference on artificial intelligence and statistics, pp 133–142

  13. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv e-prints arXiv:1412.6980

  14. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal. https://doi.org/10.1016/j.media.2017.07.005

    Article  Google Scholar 

  15. Masutani Y, Nemoto M, Nomura Y, Hayashi N (2013) Clinical machine learning in action: cad system design, development, tuning, and long-term experience. In: Suzuki K (ed) Image processing: concepts, methodologies, tools, and applications. IGI Global, Philadelphia, pp 621–638

    Chapter  Google Scholar 

  16. Metz CE (2006) Receiver operating characteristic analysis: a tool for the quantitative evaluation of observer performance and imaging systems. J Am Coll Radiol 3(6):413–22. https://doi.org/10.1016/j.jacr.2006.02.021

    Article  Google Scholar 

  17. Milletari F, Navab N, Ahmadi SA (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. arXiv e-prints arXiv:1606.04797

  18. Mockus J, Tiesis V, Zilinskas A (1978) The application of Bayesian methods for seeking the extremum. Towards Glob Optim 2:117–129

    MATH  Google Scholar 

  19. Neary P (2018) Automatic hyperparameter tuning in deep convolutional neural networks using asynchronous reinforcement learning. In: 2018 IEEE international conference on cognitive computing (ICCC), pp 73–77

  20. Nomura Y, Masutani Y, Miki S, Nemoto M, Hanaoka S, Yoshikawa T, Hayashi N, Ohtomo K (2014) Performance improvement in computerized detection of cerebral aneurysms by retraining classifier using feedback data collected in routine reading environment. J Biomed Graph Comput 4(4):12–21. https://doi.org/10.5430/jbgc.v4n4p12

    Article  Google Scholar 

  21. Nomura Y, Hayashi N, Hanaoka S, Takenaga T, Nemoto M, Miki S, Yoshikawa T, Abe O (2019) Can the spherical gold standards be used as an alternative to painted gold standards for the computerized detection of lesions using voxel-based classification? Jpn J Radiol 37(3):264–273. https://doi.org/10.1007/s11604-018-0784-6

    Article  Google Scholar 

  22. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. arXiv e-prints arXiv:1505.04597

  23. Sahiner B, Pezeshk A, Hadjiiski LM, Wang X, Drukker K, Cha KH, Summers RM, Giger ML (2019) Deep learning in medical imaging and radiation therapy. Med Phys 46(1):e1–e36. https://doi.org/10.1002/mp.13264

    Article  Google Scholar 

  24. Snoek J, Larochelle H, Adams RP (2012) Practical bayesian optimization of machine learning algorithms. In: Advances in neural information processing systems, pp 2951–2959

  25. Srinivas N, Krause A, Kakade S, Seeger M (2010) Gaussian process optimization in the bandit setting: no regret and experimental design. In: Proceedings of the 27th international conference on machine learning, Omnipress, USA, ICML’10, pp 1015–1022, http://dl.acm.org/citation.cfm?id=3104322.3104451

  26. Tokui S, Oono K, Hido S, Clayton J (2015) Chainer: a next-generation open source framework for deep learning. In: Proceedings of workshop on machine learning systems (learningsys) in the 29th annual conference on neural information processing systems (NIPS), vol 5, pp 1–6

  27. Tsujii O, Freedman MT, Mun SK (1998) Automated segmentation of anatomic regions in chest radiographs using an adaptive-sized hybrid neural network. Med Phys 25(6):998–1007. https://doi.org/10.1118/1.598277

    Article  Google Scholar 

  28. Ueda D, Shimazaki A, Miki Y (2019) Technical and clinical overview of deep learning in radiology. Jpn J Radiol 37(1):15–33. https://doi.org/10.1007/s11604-018-0795-3

    Article  Google Scholar 

  29. Wozniak JM, Jain R, Balaprakash P, Ozik J, Collier NT, Bauer J, Xia F, Brettin T, Stevens R, Mohd-Yusof J, Cardona CG, Baughman BVEM (2018) Candle/supervisor: a workflow framework for machine learning applied to cancer research. BMC Bioinform 19(18):491. https://doi.org/10.1186/s12859-018-2508-4

    Article  Google Scholar 

  30. Wu G, Zhang X, Luo S, Hu Q (2015) Lung segmentation based on customized active shape model from digital radiography chest images. J Med Imaging Health Info 5(2):184–191. https://doi.org/10.1166/jmihi.2015.1382

    Article  Google Scholar 

  31. Yasaka K, Akai H, Kunimatsu A, Kiryu S, Abe O (2018) Deep learning with convolutional neural network in radiology. Jpn J Radiol 36(4):257–272. https://doi.org/10.1007/s11604-018-0726-3

    Article  Google Scholar 

  32. Zhong Z, Zheng L, Kang G, Li S, Yang Y (2017) Random erasing data augmentation. arXiv e-prints arXiv:1708.04896

Download references

Acknowledgements

The Department of Computational Radiology and Preventive Medicine, The University of Tokyo Hospital, is sponsored by HIMEDIC Inc. and Siemens Healthcare K.K. This work was supported by the Joint Usage/Research Center for Interdisciplinary Large-scale Information Infrastructures and High Performance Computing Infrastructure projects in Japan (Project IDs: jh170036-DAH, jh180073-DAH, and jh190047-DAH).

Author information

Authors and Affiliations

  1. Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan

    Yukihiro Nomura, Tomomi Takenaga, Soichiro Miki, Takeharu Yoshikawa & Naoto Hayashi

  2. Department of Complexity Science and Engineering, Graduate School of Frontier Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656, Japan

    Issei Sato

  3. Center for Advanced Intelligence Project, RIKEN, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan

    Issei Sato

  4. Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan

    Issei Sato, Shouhei Hanaoka, Takahiro Nakao & Osamu Abe

  5. Information Technology Center, The University of Tokyo, 2-11-16 Yayoi, Bunkyo-ku, Tokyo, 113-8658, Japan

    Toshihiro Hanawa, Tetsuya Hoshino & Yuji Sekiya

  6. Department of Radiological Sciences, Ibaraki Prefectural University of Health Sciences, 4669-2 Ami, Ami-machi, Inashiki-gun, Ibaraki, 300-0394, Japan

    Tomomi Takenaga

Authors
  1. Yukihiro Nomura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Issei Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Toshihiro Hanawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shouhei Hanaoka
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Takahiro Nakao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tomomi Takenaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tetsuya Hoshino
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yuji Sekiya
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Soichiro Miki
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Takeharu Yoshikawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Naoto Hayashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Osamu Abe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yukihiro Nomura.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nomura, Y., Sato, I., Hanawa, T. et al. Development of training environment for deep learning with medical images on supercomputer system based on asynchronous parallel Bayesian optimization. J Supercomput 76, 7315–7332 (2020). https://doi.org/10.1007/s11227-020-03164-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-020-03164-7

Keywords

Search

Navigation