Abstract
Assume that we live in a deterministic world, we ask ourselves which place the device of randomness still may have, even in case that there is no philosophical incentive for it. This note argues that improved accuracy may be achieved when modeling the (deterministic) residuals of the best model of a certain complexity as ‘random’. In order to make this statement precise, the setting of adaptive compression is considered: (1) accuracy is understood in terms of codelength, and (2) the ‘random device’ relates to Solomonoff’s Algorithmic Probability (ALP) via arithmetic coding. The contribution of this letter is threefold: (a) the proposed adaptive coding scheme possesses interesting behavior in terms of its regret bound, and (b) a mathematical characterization of a deterministic world assumption is given. (c) The previous issues then facilitate the derivation of the Randomness- Complexity (RC) frontier of the given algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cesa-Bianchi, N.: Analysis of two gradient-based algorithms for on-line regression. Journal of Computer and System Sciences 59(3), 392–411 (1999)
Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press (2006)
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Springer (1991)
Csiszár, I., Shields, P.C.: Information theory and statistics: A tutorial. Now Publishers Inc. (2004)
Dowe, D.L.: MML, hybrid Bayesian network graphical models, statistical consistency, invariance and uniqueness. In: Handbook of Philosophy of Science. Handbook of Philosophy of Statistics, vol. 7, pp. 901–982. Elsevier (June 2011)
Györfi, L., Kohler, M., Krzyzak, A., Walk, H.: A distribution-free theory of nonparametric regression. Springer (2002)
Li, M., Vitányi, P.: An introduction to Kolmogorov complexity and its applications. Springer (1997)
Rissanen, J.: Modelling by shortest data description. Automatica 14, 465–471 (1978)
Shafer, G., Vovk, V.: Probability and finance: it’s only a game! Wiley Interscience (2001)
Solomonoff, R.J.: Progress in incremental machine learning. In: NIPS Workshop on Universal Learning Algorithms and Optimal Search, Whistler, BC, Canada, p. 27 (December 2002)
Wallace, C.S., Boulton, D.M.: An information measure for classification. Computer Journal 11(2), 185–194 (1968)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Pelckmans, K. (2013). An Adaptive Compression Algorithm in a Deterministic World. In: Dowe, D.L. (eds) Algorithmic Probability and Friends. Bayesian Prediction and Artificial Intelligence. Lecture Notes in Computer Science, vol 7070. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-44958-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-44958-1_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-44957-4
Online ISBN: 978-3-642-44958-1
eBook Packages: Computer ScienceComputer Science (R0)