Skip to main content
SpringerLink
Log in

Analysis on the potential of an EA–surrogate modelling tandem for deep learning parametrization: an example for cancer classification from medical images

  • S.I. : IWANN2017: Learning algorithms with real world applications
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The paper introduces a novel modality to efficiently tune the convolutional layers of a deep neural network (CNN) and an approach to also rank the importance of the involved hyperparameters. Evolutionary algorithms (EA) offer a flexible solution to this twofold issue, while the expensive simulations of the deep learner with the generated configurations are resolved by surrogate modelling. Three models have been used and evaluated as surrogates: random forests (RF), support vector machines (SVM) and Kriging. Sample convolutional configurations are generated by Latin hypercube sampling and have attached computed accuracy outcomes from real CNN runs. For the hyperparameter estimation task, the fitness of an individual from the EA associated with a surrogate model is subsequently derived from the CNN accuracy estimation on those variable values. With respect to the ranking and variable selection task, RF includes implicit variable selection, the SVM can be straightforwardly supported by a second EA, and Kriging offers a ranking based on the corresponding θ values. The estimated accuracy of the found hyperparameter values is compared with the true validation accuracy, and they are next used for the prediction on the test cases. The ranking of the variables for each of the three surrogate models is compared, and their influence is also revealed by response surface methodology. The experimental testing of the proposed EA–surrogate approaches is conducted on a real-world scenario of histopathological image interpretation in colorectal cancer diagnosis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://doi.org/10.6084/m9.figshare.4508672.v1.

References

  1. Albarqouni S, Baur C, Achilles F, Belagiannis V, Demirci S, Navab N (2016) Aggnet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans Med Imaging 35(5):1313–1321. https://doi.org/10.1109/TMI.2016.2528120

    Article  Google Scholar 

  2. Arunkumar R, Karthigaikumar P (2017) Multi-retinal disease classification by reduced deep learning features. Neural Comput Appl 28(2):329–334. https://doi.org/10.1007/s00521-015-2059-9

    Article  Google Scholar 

  3. Atencia M, Joya G, Sandoval F (2004) Parametric identification of robotic systems with stable time-varying hopfield networks. Neural Comput Appl 13(4):270–280. https://doi.org/10.1007/s00521-004-0421-4

    Article  MATH  Google Scholar 

  4. Bacciu D, Lisboa PJG, Martín JD, Stoean R, Vellido A (2018) Bioinformatics and medicine in the era of deep learning. CoRR arXiv:1802.09791

  5. Bartz-Beielstein T, Zaefferer M (2017) Model-based methods for continuous and discrete global optimization. Appl Soft Comput 55:154–167. https://doi.org/10.1016/j.asoc.2017.01.039

    Article  Google Scholar 

  6. Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. Springer, Berlin, pp 411–418

    Google Scholar 

  7. Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, Berlin

    Book  Google Scholar 

  8. Forrester AIJ, Sbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide. Wiley, New York

    Book  Google Scholar 

  9. Friese M, Bartz-Beielstein T, Emmerich M (2016) Building ensembles of surrogates by optimal convex combination. In: Papa G, Mernik M (eds) Bioinspired optimization methods and their applications. Jožef Stefan Institute, Lubljana, pp 131–143

    Google Scholar 

  10. Gorunescu F, Belciug S (2016) Boosting backpropagation algorithm by stimulus-sampling: application in computer-aided medical diagnosis. J Biomed Inform 63:74–81. https://doi.org/10.1016/j.jbi.2016.08.004

    Article  Google Scholar 

  11. Hackl C (2014) Calibration and parameterization methods for the libor market model. Springer, Berlin

    Book  Google Scholar 

  12. Heaton JB, Polson NG, Witte JH (2017) Deep learning for finance: deep portfolios. Appl Stoch Models Bus Ind 33(1):3–12. https://doi.org/10.1002/asmb.2209

    Article  MathSciNet  MATH  Google Scholar 

  13. Jiang F, Grigorev A, Rho S, Tian Z, Fu Y, Jifara W, Adil K, Liu S (2017) Medical image semantic segmentation based on deep learning. Neural Comput Appl. https://doi.org/10.1007/s00521-017-3158-6

    Article  Google Scholar 

  14. Jin Y (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evolut Comput 1(2):61–70

    Article  Google Scholar 

  15. Field RV Jr, Constantine P, Boslough M (2013) Statistical surrogate models for prediction of high-consequence climate change. Int J Uncertain Quant 3(4):341–355

    Article  MathSciNet  Google Scholar 

  16. Kapás Z, Lefkovits L, Iclanzan D, Gyorfi Á, Iantovics B, Lefkovits S, Szilágyi SM, Szilágyi L (2017) Automatic brain tumor segmentation in multispectral MRI volumes using a random forest approach. PSIVT 10749:137–149

    Google Scholar 

  17. Loshchilov I, Hutter F (2016) CMA-ES for hyperparameter optimization of deep neural networks. CoRR arXiv:1604.07269

  18. Malshe M, Narulkar R, Raff LM, Hagan M, Bukkapatnam S, Komanduri R (2008) Parametrization of analytic interatomic potential functions using neural networks. J Chem Phys 129(4):044,111. https://doi.org/10.1063/1.2957490

    Article  Google Scholar 

  19. Martens D, Baesens B, Gestel TV (2009) Decompositional rule extraction from support vector machines by active learning. IEEE Trans Knowl Data Eng 21(2):178–191. https://doi.org/10.1109/TKDE.2008.131

    Article  Google Scholar 

  20. McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245

    MathSciNet  MATH  Google Scholar 

  21. Paja W, Pancerz K (2017) Feature selection methods applied to severe brain damages data. In: 2017 Federated conference on computer science and information systems (FedCSIS), pp 199–202. https://doi.org/10.15439/2017F382

  22. Postavaru S, Stoean R, Stoean C, Joya G (2017) Adaptation of deep convolutional neural networks for cancer grading from histopathological images. In: Advances in computational intelligence: 14th international work-conference on artificial neural networks, IWANN 2017, Cadiz, Spain, 14–16 June 2017, Proceedings, Part II, Rojas, Ignacio and Joya, Gonzalo and Catala, Andreu (eds). Springer International Publishing, Cham, pp 38–49

    Chapter  Google Scholar 

  23. Preuss M (2015) Multimodal optimization by means of evolutionary algorithms. Natural computing series. Springer, Berlin. https://doi.org/10.1007/978-3-319-07407-8

    Book  MATH  Google Scholar 

  24. Sikora EBTST (2017) Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms. In: International conference on image processing. IEEE SigPort. http://sigport.org/2022

  25. Sirinukunwattana K, Raza SEA, Tsang YW, Snead DRJ, Cree IA, Rajpoot NM (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35(5):1196–1206. https://doi.org/10.1109/TMI.2016.2525803

    Article  Google Scholar 

  26. Stoean C (2016) In search of the optimal set of indicators when classifying histopathological images. In: 2016 18th International symposium on symbolic and numeric algorithms for scientific computing (SYNASC), pp 449–455. https://doi.org/10.1109/SYNASC.2016.074

  27. Stoean C, Preuss M, Stoean R (2013) EA-based parameter tuning of multimodal optimization performance by means of different surrogate models. In: Genetic and evolutionary computation conference, GECCO 2013. ACM, pp 1063–1070. https://doi.org/10.1145/2464576.2482684

  28. Stoean C, Stoean R, Sandita A, Ciobanu D, Mesina C, Gruia CL (2016) SVM-based cancer grading from histopathological images using morphological and topological features of glands and nuclei. Springer, Berlin, pp 145–155. https://doi.org/10.1007/978-3-319-39345-2_13

    Book  Google Scholar 

  29. Taylor S, Kim T, Yue Y, Mahler M, Krahe J, Rodriguez AG, Hodgins J, Matthews I (2017) A deep learning approach for generalized speech animation. ACM Trans Graph 36(4):93:1–93:11. https://doi.org/10.1145/3072959.3073699

    Article  Google Scholar 

  30. Urda D, Montes-Torres J, Moreno F, Franco L, Jerez JM (2017) Deep learning to analyze RNA-Seq gene expression data. Springer International Publishing, Berlin, pp 50–59. https://doi.org/10.1007/978-3-319-59147-6_5

    Book  Google Scholar 

  31. Volz V, Rudolph G, Naujoks B (2017) Investigating uncertainty propagation in surrogate-assisted evolutionary algorithms. In: Proceedings of the genetic and evolutionary computation conference, GECCO ’17. ACM, New York, NY, USA, pp 881–888. https://doi.org/10.1145/3071178.3071249

  32. Yoon JG, Heo J, Kim M, Park YJ, Choi MH, Song J, Wyi K, Kim H, Duchenne O, Eom S, Tsoy Y (2018) Machine learning-based diagnosis for disseminated intravascular coagulation (DIC): development, external validation, and comparison to scoring systems. PLOS ONE 13(5):1–15. https://doi.org/10.1371/journal.pone.0195861

    Article  Google Scholar 

  33. Young SR, Rose DC, Karnowski TP, Lim SH, Patton RM (2015) Optimizing deep learning hyper-parameters through an evolutionary algorithm. In: Proceedings of the workshop on machine learning in high-performance computing environments. ACM, pp 4:1–4:5. https://doi.org/10.1145/2834892.2834896

  34. Zhang J, Chowdhury S, Messac A (2012) An adaptive hybrid surrogate model. Struct Multidiscip Optim 46(2):223–238. https://doi.org/10.1007/s00158-012-0764-x

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Ruxandra Stoean

    Present address: , Craiova, Romania

Authors and Affiliations

  1. Department of Computer Science, Faculty of Sciences, University of Craiova, Craiova, Romania

    Ruxandra Stoean

Authors
  1. Ruxandra Stoean
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruxandra Stoean.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stoean, R. Analysis on the potential of an EA–surrogate modelling tandem for deep learning parametrization: an example for cancer classification from medical images. Neural Comput & Applic 32, 313–322 (2020). https://doi.org/10.1007/s00521-018-3709-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3709-5

Keywords

Search

Navigation