Abstract
The advances and development of various machine learning techniques has lead to practical solutions in various areas of science, engineering, medicine and finance. The great choice of algorithms, their implementations and libraries has resulted in another challenge of selecting the right algorithm and tuning their parameters in order to achieve optimal or satisfactory performance in specific applications. Here we show how the value of information (V(I)) can be used in this task to guide the algorithm choice and parameter tuning process. After estimating the amount of Shannon’s mutual information between the predictor and response variables, V(I) can define theoretical upper bound of performance of any algorithm. The inverse function I(V) defines the lower frontier of the minimum amount of information required to achieve the desired performance. In this paper, we illustrate the value of information for the mean-square error minimization and apply it to forecasts of cryptocurrency log-returns.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Belavkin, R.V.: Bounds of optimal learning. In: 2009 IEEE International Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pp. 199–204. IEEE, Nashville, TN, USA (2009)
Belavkin, R.V.: Information trajectory of optimal learning. In: Hirsch, M.J., Pardalos, P.M., Murphey, R. (eds.) Dynamics of Information Systems: Theory and Applications, Springer Optimization and Its Applications Series, vol. 40, pp. 29–44. Springer, New York (2010). https://doi.org/10.1007/978-1-4419-5689-7_2
Belavkin, R.V.: Mutation and optimal search of sequences in nested Hamming spaces. In: IEEE Information Theory Workshop. IEEE (2011)
Belavkin, R.V.: Dynamics of information and optimal control of mutation in evolutionary systems. In: Sorokin, A., Murphey, R., Thai, M.T., Pardalos, P.M. (eds.) Dynamics of Information Systems: Mathematical Foundations, Springer Proceedings in Mathematics and Statistics, vol. 20, pp. 3–21. Springer, New York (2012). https://doi.org/10.1007/978-1-4614-3906-6_1
Belavkin, R.V.: Optimal measures and Markov transition kernels. J. Global Optim. 55, 387–416 (2013)
Belavkin, R.V.: Asymmetry of risk and value of information. In: Vogiatzis, C., Walteros, J.L., Pardalos, P.M. (eds.) Dynamics of Information Systems. SPMS, vol. 105, pp. 1–20. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10046-3_1
Belavkin, R.V.: Relation between the Kantorovich–Wasserstein metric and the Kullback–Leibler divergence. In: Ay, N., Gibilisco, P., Matúš, F. (eds.) IGAIA IV 2016. SPMS, vol. 252, pp. 363–373. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97798-0_15
Belavkin, R.V., Channon, A., Aston, E., Aston, J., Krašovec, R., Knight, C.G.: Monotonicity of fitness landscapes and mutation rate control. J. Math. Biol. 73(6), 1491–1524 (2016)
de Jong, S.: Simpls: an alternative approach to partial least squares regression. Chemom. Intell. Lab. Syst. 18(3), 251–263 (1993)
Grishanin, B.A., Stratonovich, R.L.: Value of information and sufficient statistics during an observation of a stochastic process. Izvestiya of USSR Academy of Sciences, Technical Cybernetics 6, 4–14 (1966). in Russian
Rockafellar, R.T.: Conjugate Duality and Optimization, CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 16. Society for Industrial and Applied Mathematics, PA (1974)
Shannon, C.E.: A mathematical theory of communication. Bell System Technical Journal 27, 379–423 and 623–656 (July and October 1948)
Stratonovich, R.L.: Topics in the Theory of Random Noise, vol. 1. Martino Fine Books (2014)
Stratonovich, R.L. Theory of Information and its Value. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-22833-0
Stratonovich, R.L.: On value of information. Izvestiya of USSR Academy of Sciences, Technical Cybernetics 5, 3–12 (1965). in Russian
Stratonovich, R.L.: Value of information during an observation of a stochastic process in systems with finite state automata. Izvestiya of USSR Academy of Sciences, Technical Cybernetics 5, 3–13 (1966). in Russian
Stratonovich, R.L.: Extreme problems of information theory and dynamic programming. Izvestiya of USSR Academy of Sciences, Technical Cybernetics 5, 63–77 (1967). in Russian
Stratonovich, R.L.: Theory of Information. Sovetskoe Radio, Moscow, USSR (1975). in Russian
Stratonovich, R.L., Grishanin, B.A.: Value of information when an estimated random variable is hidden. Izvestiya of USSR Academy of Sciences, Technical Cybernetics 3, 3–15 (1966). in Russian
Stratonovich, R.L., Grishanin, B.A.: Game-theoretic problems with information constraints. Izvestiya of USSR Academy of Sciences, Technical Cybernetics 1, 3–12 (1968). in Russian
Tikhomirov, V.M.: Analysis II, Encyclopedia of Mathematical Sciences, vol. 14, chap. Convex Analysis, pp. 1–92. Springer-Verlag (1990)
Acknowledgements
Stefan Behringer is deeply acknowledged for additional discussion of the example, Roman Tarabrin is deeply acknowledged for providing a MacBookPro laptop used for the computational experiments. This research was funded in part by the ONR grant number N00014-21-1-2295.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Belavkin, R.V., Pardalos, P., Principe, J. (2022). Value of Information in the Mean-Square Case and Its Application to the Analysis of Financial Time-Series Forecast. In: Simos, D.E., Rasskazova, V.A., Archetti, F., Kotsireas, I.S., Pardalos, P.M. (eds) Learning and Intelligent Optimization. LION 2022. Lecture Notes in Computer Science, vol 13621. Springer, Cham. https://doi.org/10.1007/978-3-031-24866-5_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-24866-5_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-24865-8
Online ISBN: 978-3-031-24866-5
eBook Packages: Computer ScienceComputer Science (R0)