Abstract
The new 5G mobile communication system era brings a new set of communication devices that will appear on the market. These devices will generate data streams that require proper handling by machine algorithms. The processing of these data streams requires the design, development, and adaptation of appropriate machine learning algorithms. While stream processing algorithms include hyper-parameters for performance refinement, their tuning process is time-consuming and typically requires an expert to do the task.
In this paper, we present an extension of the Self Parameter Tuning (SPT) optimization algorithm for data streams. We apply the Nelder-Mead algorithm to dynamically sized samples that converge to optimal settings in a double pass over data (during the exploration phase), using a relatively small number of data points. Additionally, the SPT automatically readjusts hyper-parameters when concept drift occurs.
We did a set of experiments with well-known classification data sets and the results show that the proposed algorithm can outperform the results of previous hyper-parameter tuning efforts by human experts. The statistical results show that this extension is faster in terms of convergence and presents at least similar accuracy results when compared with the standard optimization techniques.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The source code is available on https://github.com/BrunoMVeloso/SPT/blob/master/IoTStream2020.zip – The password of the source file is “SPT”.
- 2.
- 3.
- 4.
- 5.
References
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012). https://doi.org/10.5555/2188385.2188395
Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: Moa: massive online analysis. J. Mach. Learn. Res. 11, 1601–1604 (2010). https://doi.org/10.5555/1756006.1859903
Brazdil, P.B., Soares, C., da Costa, J.P.: Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results. Mach. Learn. 50(3), 251–277 (2003). https://doi.org/10.1023/A:1021713901879
Elshawi, R., Maher, M., Sakr, S.: Automated machine learning: state-of-the-art and open challenges (2019)
Feurer, M., Hutter, F.: Hyperparameter Optimization, pp. 3–33. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-05318-5_1
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1126–1135. PMLR, International Convention Centre, Sydney, Australia, 06–11 August 2017. https://doi.org/10.5555/3305381.3305498
Kar, R., Konar, A., Chakraborty, A., Ralescu, A.L., Nagar, A.K.: Extending the nelder-mead algorithm for feature selection from brain networks. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 4528–4534, July 2016. https://doi.org/10.1109/CEC.2016.7744366
Koenigstein, N., Dror, G., Koren, Y.: Yahoo! music recommendations: modeling music ratings with temporal dynamics and item taxonomy. In: Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys 2011, pp. 165–172. ACM, New York (2011). https://doi.org/10.1145/2043932.2043964
Kohavi, R., John, G.H.: Automatic parameter selection by minimizing estimated error. In: Prieditis, A., Russell, S. (eds.) Machine Learning Proceedings 1995, pp. 304–312. Morgan Kaufmann, San Francisco (CA) (1995). https://doi.org/10.1016/B978-1-55860-377-6.50045-1
Lerman, P.M.: Fitting segmented regression models by grid search. J. Royal Stat. Soc.: Ser. C (Appl. Stat.) 29(1), 77–84 (1980). https://doi.org/10.2307/2346413
Maclaurin, D., Duvenaud, D., Adams, R.P.: Gradient-based hyperparameter optimization through reversible learning. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, ICML2015, vol. 37, pp. 2113–2122. JMLR.org (2015). https://doi.org/10.5555/3045118.3045343
Manapragada, C., Webb, G.I., Salehi, M.: Extremely fast decision tree. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1953–1962 (2018)
Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965). https://doi.org/10.1093/comjnl/7.4.308
Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms. CoRR abs/1803.02999 (2018)
Pfaffe, P., Tillmann, M., Walter, S., Tichy, W.F.: Online-autotuning in the presence of algorithmic choice. In: 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 1379–1388, May 2017. https://doi.org/10.1109/IPDPSW.2017.28
da Silva Fernandes, S., Tork, H.F., da Gama, J.M.P.: The initialization and parameter setting problem in tensor decomposition-based link prediction. In: 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 99–108, October 2017. https://doi.org/10.1109/DSAA.2017.83
Veloso, B., Gama, J., Malheiro, B.: Self hyper-parameter tuning for data streams. In: Soldatova, L., Vanschoren, J., Papadopoulos, G., Ceci, M. (eds.) Discovery Science, pp. 241–255. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01771-2_16
Veloso, B., Gama, J., Malheiro, B., Vinagre, J.: Self hyper-parameter tuning for stream recommendation algorithms. In: Monreale, A., et al. (eds.) ECML PKDD 2018 Workshops, pp. 91–102. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14880-5_8
Acknowledgments
This research was Funded from national funds through FCT - Science and Technology Foundation, I.P in the context of the project FailStopper (DSAIPA/DS/0086/2018).
This work is financed by National Funds through the Portuguese funding agency, FCT - Fundação para a Ciência e a Tecnologia, within project UIDB/50014/2020.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Veloso, B., Gama, J. (2020). Self Hyper-parameter Tuning for Stream Classification Algorithms. In: Gama, J., et al. IoT Streams for Data-Driven Predictive Maintenance and IoT, Edge, and Mobile for Embedded Machine Learning. ITEM IoT Streams 2020 2020. Communications in Computer and Information Science, vol 1325. Springer, Cham. https://doi.org/10.1007/978-3-030-66770-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-66770-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66769-6
Online ISBN: 978-3-030-66770-2
eBook Packages: Computer ScienceComputer Science (R0)