Skip to main content
SpringerLink
Log in

RADSSo: An Automated Tool for the multi-CASH Machine Learning Problem

  • Conference paper
  • First Online:
Hybrid Artificial Intelligent Systems (HAIS 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12344))

Included in the following conference series:

  • 991 Accesses

Abstract

The increasing application of machine learning techniques to different disciplines has driven the research in the field towards the creation of algorithms able to construct the best model with the optimal hyperparameter configuration for a particular problem, without the need of user’s expert knowledge. This is well-known as the Combined Algorithms Selection and Hyperparameter Optimization problem.

In this work, we develop the open-source tool RADSSo in order to solve a multi-scenario of combined algorithms selection and hyperparameter optimization in which only one datastore is available containing many different machine learning problems. Then, several models need to be computed, at the same time, in an automated way. The tool is deployed in a modular form that allows to modify it and to customize the configuration files to adapt the tool to any context.

The underlying model is a mathematical formula that scores each machine learning model providing the best one for each subsample of the datastore. This score is based on the suitability of the model, different metrics from the confusion matrix and the capability of the generalization by the learning curves. In addition, RADSSo provides intuitively reports with all the essential information.

Partially supported by the Spanish National Cybersecurity Institute (INCIBE) under contracts art.83, keys: X43 and X54.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Research Institute of Applied Sciences in Cybersecurity (RIASC).

References

  1. KNIME AG: KNIME software. https://www.knime.com

  2. Aladag, C.H., Egrioglu, E., Gunay, S., Basaran, M.A.: Improving weighted information criterion by using optimization. J. Comput. Appl. Math. 233(10), 2683–2687 (2010). https://doi.org/10.1016/j.cam.2009.11.016

    Article  MathSciNet  MATH  Google Scholar 

  3. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012)

    MathSciNet  MATH  Google Scholar 

  4. Brochu, E., Cora, V.M., de Freitas, N.: A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. The Computing Research Repository (CoRR) (2010). http://arxiv.org/abs/1012.2599

  5. DeCastro-García, N., Muñoz Castañeda, Á.L., Escudero García, D., Carriegos, M.V.: Effect of the sampling of a dataset in the hyperparameter optimization phase over the efficiency of a machine learning algorithm. Complexity 2019 (2019). https://doi.org/10.1155/2019/6278908

  6. DeCastro-García, N., Muñoz Castañeda, Á.L., Fernández-Rodríguez, M.: Machine learning for automatic assignment of the severity of cybersecurity events. Comput. Math. Meth. 2(1), e1072 (2020). https://doi.org/10.1002/cmm4.1072

    Article  Google Scholar 

  7. Feurer, M., Klein, A., Eggensperger, K., Springenberg, J.T., Blum, M., Hutter, F.: Efficient and robust automated machine learning. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, NIPS 2015, vol. 2, pp. 2755–2763. MIT Press, Cambridge (2015)

    Google Scholar 

  8. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. ACM SIGKDD Explor. Newslett. 11(1), 10–18 (2009). https://doi.org/10.1145/1656274.1656278

    Article  Google Scholar 

  9. Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25566-3_40

    Chapter  Google Scholar 

  10. Komer, B., Bergstra, J., Eliasmith, C.: Hyperopt-sklearn: automatic hyperparameter configuration for scikit-learn. In: Proceedings of the 13th Python in Science Conference, SCIPY 2014, pp. 32–37 (2014). https://doi.org/10.25080/Majora-14bd3278-006

  11. Kotthoff, L., Thornton, C., Hoos, H.H., Hutter, F., Leyton-Brown, K.: Auto-WEKA 2.0: automatic model selection and hyperparameter optimization in WEKA. J. Mach. Learn. Res. 18(25), 1–5 (2017)

    Google Scholar 

  12. Kozachenko, L.F., Leonenko, N.N.: Sample estimate of the entropy of a random vector. Probl. Inf. Trans. 23(2), 95–101 (1987)

    MATH  Google Scholar 

  13. Kraskov, A., Stögbauer, H., Grassberger, P.: Estimating mutual information. Phys. Rev. E 69, 066138 (2004). https://doi.org/10.1103/PhysRevE.69.066138

    Article  MathSciNet  Google Scholar 

  14. Matthews, B.: Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA) Protein Struct. 405(2), 442–451 (1975). https://doi.org/10.1016/0005-2795(75)90109-9

  15. Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: Yale: rapid prototyping for complex data mining tasks. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2006, pp. 935–940. ACM, New York (2006). https://doi.org/10.1145/1150402.1150531

  16. Muñoz Castañeda, Á.L., Escudero García, D., DeCastro-García, N., Carriegos, M.V.: RIASC hyperparameter optimization automated software. https://github.com/amunc/RHOASo

  17. Nuñez, L., Regis, R.G., Varela, K.: Accelerated random search for constrained global optimization assisted by radial basis function surrogates. J. Comput. Appl. Math. 340, 276–295 (2018). https://doi.org/10.1016/j.cam.2018.02.017

    Article  MathSciNet  MATH  Google Scholar 

  18. Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  19. RapidMiner, I.: Rapidminer. https://rapidminer.com

  20. Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2013, pp. 847–855. ACM, New York (2013). https://doi.org/10.1145/2487575.2487629

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Universidad de León, León, Spain

    Noemí DeCastro-García & Ángel Luis Muñoz Castañeda

  2. Research Institute of Applied Sciences in Cybersecurity (RIASC), Universidad de León, León, Spain

    Mario Fernández-Rodríguez

Authors
  1. Noemí DeCastro-García
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ángel Luis Muñoz Castañeda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario Fernández-Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noemí DeCastro-García .

Editor information

Editors and Affiliations

  1. University of Oviedo, Oviedo, Spain

    Enrique Antonio de la Cal

  2. University of Oviedo, Oviedo, Spain

    José Ramón Villar Flecha

  3. University of A Coruña, Ferrol, Spain

    Héctor Quintián

  4. University of Salamanca, Salamanca, Spain

    Emilio Corchado

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

DeCastro-García, N., Castañeda, Á.L.M., Fernández-Rodríguez, M. (2020). RADSSo: An Automated Tool for the multi-CASH Machine Learning Problem. In: de la Cal, E.A., Villar Flecha, J.R., Quintián, H., Corchado, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2020. Lecture Notes in Computer Science(), vol 12344. Springer, Cham. https://doi.org/10.1007/978-3-030-61705-9_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61705-9_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61704-2

  • Online ISBN: 978-3-030-61705-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation