Abstract
Given a set X of sequences over a finite alphabet, we investigate the following three quantities.
-
(i)
The feasible predictability of X is the highest success ratio that a polynomial-time randomized predictor can achieve on all sequences in X.
-
(ii)
The deterministic feasible predictability of X is the highest success ratio that a polynomial-time deterministic predictor can achieve on all sequences in X.
-
(iii)
The feasible dimension of X is the polynomial-time effectivization of the classical Hausdorff dimension (“fractal dimension”) of X.
Predictability is known to be stable in the sense that the feasible predictability of X∪Y is always the minimum of the feasible predictabilities of X and Y. We show that deterministic predictability also has this property if X and Y are computably presentable. We show that deterministic predictability coincides with predictability on singleton sets. Our main theorem states that the feasible dimension of X is bounded above by the maximum entropy of the predictability of X and bounded below by the segmented self-information of the predictability of X, and that these bounds are tight.
This author’s research was supported in part by National Science Foundation Grant CCR-9988483. Much of the work was done while this author was on sabbatical at NEC Research Institute.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
J. L. Balcázar, J. Díaz, and J. Gabarró. Structural Complexity I (second edition). Springer-Verlag, Berlin, 1995.
N. Cesa-Bianchi, Y. Freund, D. Haussler, D. P. Helmbold, R. E. Schapire, and M. K. Warmuth. How to use expert advice. Journal of the ACM, 44(3):427–485, May 1997.
N. Cesa-Bianchi and G. Lugosi. On prediction of individual sequences. Annals of Statistics, 27(6):1865–1895, 1999.
T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley & Sons, Inc., New York, N.Y., 1991.
J. J. Dai, J. I. Lathrop, J. H. Lutz, and E. Mayordomo. Finite-state dimension. In Proceedings of the Twenty-Eighth International Colloquium on Automata, Languages, and Programming, pages 1028–1039. Springer-Verlag, 2001.
K. Falconer. Fractal Geometry: Mathematical Foundations and Applications. John Wiley & Sons, 1990.
M. Feder. Gambling using a finite state machine. IEEE Transactions on Information Theory, 37:1459–1461, 1991.
M. Feder, N. Merhav, and M. Gutman. Universal prediction of individual sequences. IEEE Transations on Information Theory, 38:1258–1270, 1992.
F. Hausdorff. Dimension und äusseres Mass. Math. Ann., 79:157–179, 1919.
J. Kelly. A new interpretation of information rate. Bell Systems Technical Journal, 35:917–926, 1956.
A. N. Kolmogorov. On tables of random numbers. Sankhyā, Series A, 25:369–376, 1963.
D. W. Loveland. A new interpretation of von Mises’ concept of a random sequence. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 12:279–294, 1966.
J. H. Lutz. Dimension in complexity classes. In Proceedings of the Fifteenth Annual IEEE Conference on Computational Complexity, pages 158–169. IEEE Computer Society Press, 2000. Updated version appears as Technical report cs.CC/0203016, ACM Computing Research Repository, 2002.
J. H. Lutz. Gales and the constructive dimension of individual sequences. In Proceedings of the Twenty-Seventh International Colloquium on Automata, Languages, and Programming, pages 902–913. Springer-Verlag, 2000. Updated version appears as Technical report cs.CC/0203017, ACM Computing Research Repository, 2002.
D. A. Martin. Classes of recursively enumerable sets and degrees of unsolvability. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 12:295–310, 1966.
B. Ya. Ryabko. Noiseless coding of combinatorial sources. Problems of Information Transmission, 22:170–179, 1986.
B. Ya. Ryabko. Algorithmic approach to the prediction problem. Problems of Information Transmission, 29:186–193, 1993.
B. Ya. Ryabko. The complexity and effectiveness of prediction problems. Journal of Complexity, 10:281–295, 1994.
C. P. Schnorr. A unified approach to the definition of random sequences. Mathematical Systems Theory, 5:246–258, 1971.
C. P. Schnorr. Zufälligkeit und Wahrscheinlichkeit. Lecture Notes in Mathematics, 218, 1971.
C. E. Shannon. Certain results in coding theory for noisy channels. Bell Systems Technical Journal, 30:50–64, 1951.
A. Kh. Shen’. On relations between different algorithmic definitions of randomness. Soviet Mathematics Doklady, 38:316–319, 1989.
L. Staiger. Kolmogorov complexity and Hausdorff dimension. Information and Control, 103:159–94, 1993.
L. Staiger. A tight upper bound on Kolmogorov complexity and uniformly optimal prediction. Theory of Computing Systems, 31:215–29, 1998.
V. Vovk. A game of prediction with expert advice. Journal of Computer and System Sciences, pages 153–173, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fortnow, L., Lutz, J.H. (2002). Prediction and Dimension. In: Kivinen, J., Sloan, R.H. (eds) Computational Learning Theory. COLT 2002. Lecture Notes in Computer Science(), vol 2375. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45435-7_26
Download citation
DOI: https://doi.org/10.1007/3-540-45435-7_26
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43836-6
Online ISBN: 978-3-540-45435-9
eBook Packages: Springer Book Archive