Skip to main content

Prediction and Dimension

  • Conference paper
  • First Online:
Computational Learning Theory (COLT 2002)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2375))

Included in the following conference series:

  • 1199 Accesses

Abstract

Given a set X of sequences over a finite alphabet, we investigate the following three quantities.

  1. (i)

    The feasible predictability of X is the highest success ratio that a polynomial-time randomized predictor can achieve on all sequences in X.

  2. (ii)

    The deterministic feasible predictability of X is the highest success ratio that a polynomial-time deterministic predictor can achieve on all sequences in X.

  3. (iii)

    The feasible dimension of X is the polynomial-time effectivization of the classical Hausdorff dimension (“fractal dimension”) of X.

Predictability is known to be stable in the sense that the feasible predictability of XY is always the minimum of the feasible predictabilities of X and Y. We show that deterministic predictability also has this property if X and Y are computably presentable. We show that deterministic predictability coincides with predictability on singleton sets. Our main theorem states that the feasible dimension of X is bounded above by the maximum entropy of the predictability of X and bounded below by the segmented self-information of the predictability of X, and that these bounds are tight.

This author’s research was supported in part by National Science Foundation Grant CCR-9988483. Much of the work was done while this author was on sabbatical at NEC Research Institute.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. J. L. Balcázar, J. Díaz, and J. Gabarró. Structural Complexity I (second edition). Springer-Verlag, Berlin, 1995.

    Google Scholar 

  2. N. Cesa-Bianchi, Y. Freund, D. Haussler, D. P. Helmbold, R. E. Schapire, and M. K. Warmuth. How to use expert advice. Journal of the ACM, 44(3):427–485, May 1997.

    Google Scholar 

  3. N. Cesa-Bianchi and G. Lugosi. On prediction of individual sequences. Annals of Statistics, 27(6):1865–1895, 1999.

    Article  MATH  MathSciNet  Google Scholar 

  4. T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley & Sons, Inc., New York, N.Y., 1991.

    MATH  Google Scholar 

  5. J. J. Dai, J. I. Lathrop, J. H. Lutz, and E. Mayordomo. Finite-state dimension. In Proceedings of the Twenty-Eighth International Colloquium on Automata, Languages, and Programming, pages 1028–1039. Springer-Verlag, 2001.

    Google Scholar 

  6. K. Falconer. Fractal Geometry: Mathematical Foundations and Applications. John Wiley & Sons, 1990.

    Google Scholar 

  7. M. Feder. Gambling using a finite state machine. IEEE Transactions on Information Theory, 37:1459–1461, 1991.

    Article  MathSciNet  Google Scholar 

  8. M. Feder, N. Merhav, and M. Gutman. Universal prediction of individual sequences. IEEE Transations on Information Theory, 38:1258–1270, 1992.

    Article  MATH  MathSciNet  Google Scholar 

  9. F. Hausdorff. Dimension und äusseres Mass. Math. Ann., 79:157–179, 1919.

    Article  MathSciNet  Google Scholar 

  10. J. Kelly. A new interpretation of information rate. Bell Systems Technical Journal, 35:917–926, 1956.

    Google Scholar 

  11. A. N. Kolmogorov. On tables of random numbers. Sankhyā, Series A, 25:369–376, 1963.

    MATH  MathSciNet  Google Scholar 

  12. D. W. Loveland. A new interpretation of von Mises’ concept of a random sequence. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 12:279–294, 1966.

    Article  MATH  MathSciNet  Google Scholar 

  13. J. H. Lutz. Dimension in complexity classes. In Proceedings of the Fifteenth Annual IEEE Conference on Computational Complexity, pages 158–169. IEEE Computer Society Press, 2000. Updated version appears as Technical report cs.CC/0203016, ACM Computing Research Repository, 2002.

    Google Scholar 

  14. J. H. Lutz. Gales and the constructive dimension of individual sequences. In Proceedings of the Twenty-Seventh International Colloquium on Automata, Languages, and Programming, pages 902–913. Springer-Verlag, 2000. Updated version appears as Technical report cs.CC/0203017, ACM Computing Research Repository, 2002.

    Google Scholar 

  15. D. A. Martin. Classes of recursively enumerable sets and degrees of unsolvability. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 12:295–310, 1966.

    Article  MATH  Google Scholar 

  16. B. Ya. Ryabko. Noiseless coding of combinatorial sources. Problems of Information Transmission, 22:170–179, 1986.

    MATH  MathSciNet  Google Scholar 

  17. B. Ya. Ryabko. Algorithmic approach to the prediction problem. Problems of Information Transmission, 29:186–193, 1993.

    MathSciNet  Google Scholar 

  18. B. Ya. Ryabko. The complexity and effectiveness of prediction problems. Journal of Complexity, 10:281–295, 1994.

    Article  MATH  MathSciNet  Google Scholar 

  19. C. P. Schnorr. A unified approach to the definition of random sequences. Mathematical Systems Theory, 5:246–258, 1971.

    Article  MATH  MathSciNet  Google Scholar 

  20. C. P. Schnorr. Zufälligkeit und Wahrscheinlichkeit. Lecture Notes in Mathematics, 218, 1971.

    Google Scholar 

  21. C. E. Shannon. Certain results in coding theory for noisy channels. Bell Systems Technical Journal, 30:50–64, 1951.

    Google Scholar 

  22. A. Kh. Shen’. On relations between different algorithmic definitions of randomness. Soviet Mathematics Doklady, 38:316–319, 1989.

    MATH  MathSciNet  Google Scholar 

  23. L. Staiger. Kolmogorov complexity and Hausdorff dimension. Information and Control, 103:159–94, 1993.

    MATH  MathSciNet  Google Scholar 

  24. L. Staiger. A tight upper bound on Kolmogorov complexity and uniformly optimal prediction. Theory of Computing Systems, 31:215–29, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  25. V. Vovk. A game of prediction with expert advice. Journal of Computer and System Sciences, pages 153–173, 1998.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fortnow, L., Lutz, J.H. (2002). Prediction and Dimension. In: Kivinen, J., Sloan, R.H. (eds) Computational Learning Theory. COLT 2002. Lecture Notes in Computer Science(), vol 2375. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45435-7_26

Download citation

  • DOI: https://doi.org/10.1007/3-540-45435-7_26

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43836-6

  • Online ISBN: 978-3-540-45435-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics