Abstract
We will call a set of programs a Prediction Tool (PT) that can be used to solve a particular applied prediction problem, for example, predicting the volumes of traffic under consideration at certain points in the future. The goal may be a forecast for the network administrator. We analyze what kind of information about the predicted data and the predictors should be used to develop (design) PT. The paper analyzes some principal questions, the solution of which is essential for specified procedures of choosing a predictor in the prediction online scheme. This is primarily a question about the properties of predictability of random sequences, and the required and achievable accuracy of the estimate of the conditional probability of prediction obtained from past results. Although some of these issues have been considered in sufficient detail in the literature, for example, such as the analysis of predictability measures, accuracy metrics, however, as will be shown, they are more focused on the problems of constructing specific prediction algorithms than on the choice of ready-made predictors.
It is shown how the specified properties of sequences and probability estimates affect the quality of the choice of predictors. Based on this analysis, a rule for choosing a predictor based on the results of previous predictions is formulated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Feder, M., Merhav, N., Gutman, M.: Universal prediction of individual sequences. IEEE Trans. Inf. Theory 38(4), 887–892 (1992)
Ryabko, B.B.: Compression-based methods for nonparametric prediction and estimation of some characteristics of time series. IEEE Trans. Inf. Theor. 55(9), 4309–4315 (2009)
Feder, M., Merhav, N.: Universal prediction. IEEE Trans. Inf. Theor. 44(6), 2124–2147 (1998)
Meron, E., Feder, M.: Finite-memory universal prediction of individual sequences. IEEE Trans. Inf. Theor. 50(7), 1506–1523 (2004)
Lee, Y.K., Lee, E.R., Park, B.U.: Conditional quantile estimation by local logistic regression. Nonparametric Stat. 4(6), 357–373 (2006)
Stoltz, G.: Incomplete information and internal regret in prediction of individual sequences (2005). https://tel.archives-ouvertes.fr/tel-00009759
Lysyak, A.S., Ryabko, B.Y.: Time series prediction based on data compression methods. Probl. Inf. Transm. 52(1), 92–99 (2016). https://doi.org/10.1134/S0032946016010075
Chen, T., Guestrin, C.: XGboost: a scalable tree boosting system. arXiv.org (2016). https://arxiv.org/pdf/1603.02754.pdf
Buczak, L., Guven, E.: A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Com. Surv. Tut. 18(2), 1153–1176 (2016)
Rooba, R., Vallimayil, V.: Semantic aware future page prediction based on domain. Int. J. Pure Appl. Math. 118(9), 911–919 (2018)
Ryabko, B., Monarev, V.: Using information theory approach to randomness testing. J. Stat. Plann. Infer. 133, 95–110 (2005)
Frenkel, S.: Ontological and probabilistic aspects of assessing predictors quality. In: Book of Abstract of VII Workshop on Computational Data Analysis and Numerical Methods Polytechnic Institute of Tomar, Portugal, pp. 84–86 (2020)
Frenkel, S, Zakharov, V., Basok, M: Optimization of the integration process of Cloud and COTS based computing systems. In: Posin, B. (ed.) Proceedings of VI International Conference program Actual Problems of System and Software Engineering (APSSE 2019), pp. 88–94. IEEE Computer Society, Moscow (2019)
Acknowledgements
Research partially supported by the Russian Foundation for Basic Research under grants RFBR 18-29-03100.
I express my gratitude to Professor Shlomi Dolev (Ben-Gurion University of the Negev) for a fruitful discussion of the predictability issues.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Frenkel, S. (2021). Theoretical Aspects of a Priori On-Line Assessment of Data Predictability in Applied Tasks. In: Dolev, S., Margalit, O., Pinkas, B., Schwarzmann, A. (eds) Cyber Security Cryptography and Machine Learning. CSCML 2021. Lecture Notes in Computer Science(), vol 12716. Springer, Cham. https://doi.org/10.1007/978-3-030-78086-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-78086-9_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78085-2
Online ISBN: 978-3-030-78086-9
eBook Packages: Computer ScienceComputer Science (R0)