Skip to main content
SpringerLink
Log in

Self Hyper-parameter Tuning for Stream Classification Algorithms

  • Conference paper
  • First Online:
Book cover IoT Streams for Data-Driven Predictive Maintenance and IoT, Edge, and Mobile for Embedded Machine Learning (ITEM 2020, IoT Streams 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1325))

Abstract

The new 5G mobile communication system era brings a new set of communication devices that will appear on the market. These devices will generate data streams that require proper handling by machine algorithms. The processing of these data streams requires the design, development, and adaptation of appropriate machine learning algorithms. While stream processing algorithms include hyper-parameters for performance refinement, their tuning process is time-consuming and typically requires an expert to do the task.

In this paper, we present an extension of the Self Parameter Tuning (SPT) optimization algorithm for data streams. We apply the Nelder-Mead algorithm to dynamically sized samples that converge to optimal settings in a double pass over data (during the exploration phase), using a relatively small number of data points. Additionally, the SPT automatically readjusts hyper-parameters when concept drift occurs.

We did a set of experiments with well-known classification data sets and the results show that the proposed algorithm can outperform the results of previous hyper-parameter tuning efforts by human experts. The statistical results show that this extension is faster in terms of convergence and presents at least similar accuracy results when compared with the standard optimization techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The source code is available on https://github.com/BrunoMVeloso/SPT/blob/master/IoTStream2020.zip – The password of the source file is “SPT”.

  2. 2.

    https://datahub.io/machine-learning/electricity#resource-electricity_arff.

  3. 3.

    https://archive.ics.uci.edu/ml/datasets/Avila.

  4. 4.

    http://www.liaad.up.pt/kdus/products/datasets-for-concept-drift.

  5. 5.

    https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients.

References

  1. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012). https://doi.org/10.5555/2188385.2188395

    Article  MathSciNet  MATH  Google Scholar 

  2. Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: Moa: massive online analysis. J. Mach. Learn. Res. 11, 1601–1604 (2010). https://doi.org/10.5555/1756006.1859903

    Article  Google Scholar 

  3. Brazdil, P.B., Soares, C., da Costa, J.P.: Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results. Mach. Learn. 50(3), 251–277 (2003). https://doi.org/10.1023/A:1021713901879

    Article  MATH  Google Scholar 

  4. Elshawi, R., Maher, M., Sakr, S.: Automated machine learning: state-of-the-art and open challenges (2019)

    Google Scholar 

  5. Feurer, M., Hutter, F.: Hyperparameter Optimization, pp. 3–33. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-05318-5_1

  6. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1126–1135. PMLR, International Convention Centre, Sydney, Australia, 06–11 August 2017. https://doi.org/10.5555/3305381.3305498

  7. Kar, R., Konar, A., Chakraborty, A., Ralescu, A.L., Nagar, A.K.: Extending the nelder-mead algorithm for feature selection from brain networks. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 4528–4534, July 2016. https://doi.org/10.1109/CEC.2016.7744366

  8. Koenigstein, N., Dror, G., Koren, Y.: Yahoo! music recommendations: modeling music ratings with temporal dynamics and item taxonomy. In: Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys 2011, pp. 165–172. ACM, New York (2011). https://doi.org/10.1145/2043932.2043964

  9. Kohavi, R., John, G.H.: Automatic parameter selection by minimizing estimated error. In: Prieditis, A., Russell, S. (eds.) Machine Learning Proceedings 1995, pp. 304–312. Morgan Kaufmann, San Francisco (CA) (1995). https://doi.org/10.1016/B978-1-55860-377-6.50045-1

  10. Lerman, P.M.: Fitting segmented regression models by grid search. J. Royal Stat. Soc.: Ser. C (Appl. Stat.) 29(1), 77–84 (1980). https://doi.org/10.2307/2346413

    Article  Google Scholar 

  11. Maclaurin, D., Duvenaud, D., Adams, R.P.: Gradient-based hyperparameter optimization through reversible learning. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, ICML2015, vol. 37, pp. 2113–2122. JMLR.org (2015). https://doi.org/10.5555/3045118.3045343

  12. Manapragada, C., Webb, G.I., Salehi, M.: Extremely fast decision tree. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1953–1962 (2018)

    Google Scholar 

  13. Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965). https://doi.org/10.1093/comjnl/7.4.308

  14. Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms. CoRR abs/1803.02999 (2018)

    Google Scholar 

  15. Pfaffe, P., Tillmann, M., Walter, S., Tichy, W.F.: Online-autotuning in the presence of algorithmic choice. In: 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 1379–1388, May 2017. https://doi.org/10.1109/IPDPSW.2017.28

  16. da Silva Fernandes, S., Tork, H.F., da Gama, J.M.P.: The initialization and parameter setting problem in tensor decomposition-based link prediction. In: 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 99–108, October 2017. https://doi.org/10.1109/DSAA.2017.83

  17. Veloso, B., Gama, J., Malheiro, B.: Self hyper-parameter tuning for data streams. In: Soldatova, L., Vanschoren, J., Papadopoulos, G., Ceci, M. (eds.) Discovery Science, pp. 241–255. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01771-2_16

  18. Veloso, B., Gama, J., Malheiro, B., Vinagre, J.: Self hyper-parameter tuning for stream recommendation algorithms. In: Monreale, A., et al. (eds.) ECML PKDD 2018 Workshops, pp. 91–102. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14880-5_8

Download references

Acknowledgments

This research was Funded from national funds through FCT - Science and Technology Foundation, I.P in the context of the project FailStopper (DSAIPA/DS/0086/2018).

This work is financed by National Funds through the Portuguese funding agency, FCT - Fundação para a Ciência e a Tecnologia, within project UIDB/50014/2020.

Author information

Authors and Affiliations

  1. FEP, University of Porto, Porto, Portugal

    João Gama

  2. INESC TEC, Porto, Portugal

    Bruno Veloso & João Gama

  3. Universidade Portucalense, Porto, Portugal

    Bruno Veloso

Authors
  1. Bruno Veloso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Gama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bruno Veloso .

Editor information

Editors and Affiliations

  1. University of Porto, Porto, Portugal

    Joao Gama

  2. Halmstad University, Halmstad, Sweden

    Sepideh Pashami

  3. Waikato University, Hamilton, New Zealand

    Albert Bifet

  4. University of Lille, Lille, France

    Moamar Sayed-Mouchawe

  5. Heidelberg University, Heidelberg, Germany

    Holger Fröning

  6. Graz University of Technology, Graz, Austria

    Franz Pernkopf

  7. University of Duisburg-Essen, Essen, Germany

    Gregor Schiele

  8. XILINX Research, Dublin, Ireland

    Michaela Blott

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Veloso, B., Gama, J. (2020). Self Hyper-parameter Tuning for Stream Classification Algorithms. In: Gama, J., et al. IoT Streams for Data-Driven Predictive Maintenance and IoT, Edge, and Mobile for Embedded Machine Learning. ITEM IoT Streams 2020 2020. Communications in Computer and Information Science, vol 1325. Springer, Cham. https://doi.org/10.1007/978-3-030-66770-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-66770-2_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-66769-6

  • Online ISBN: 978-3-030-66770-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation