Abstract
If S is an infinite sequence over a finite alphabet Σ and β is a probability measure on Σ, then the dimension of S with respect to β, written \(\dim^\beta(S)\), is a constructive version of Billingsley dimension that coincides with the (constructive Hausdorff) dimension dim(S) when β is the uniform probability measure. This paper shows that dimβ(S) and its dual Dimβ(S), the strong dimension of S with respect to β, can be used in conjunction with randomness to measure the similarity of two probability measures α and β on Σ. Specifically, we prove that the divergence formula
holds whenever α and β are computable, positive probability measures on Σ and R ∈ Σ ∞ is random with respect to α. In this formula, \({\mathcal{H}}(\alpha)\) is the Shannon entropy of α, and \({\mathcal{D}}(\alpha||\beta)\) is the Kullback-Leibler divergence between α and β.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Athreya, K.B., Hitchcock, J.M., Lutz, J.H., Mayordomo, E.: Effective strong dimension, algorithmic information, and computational complexity. SIAM Journal on Computing 37, 671–705 (2007)
Billingsley, P.: Hausdorff dimension in probability theory. Illinois Journal of Mathematics 4, 187–209 (1960)
Borel, E.: Sur les probabilités dénombrables et leurs applications arithmétiques. Rend. Circ. Mat. Palermo 27, 247–271 (1909)
Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. John Wiley & Sons, Inc., Chichester (2006)
Dai, J.J., Lathrop, J.I., Lutz, J.H., Mayordomo, E.: Finite-state dimension. Theoretical Computer Science 310, 1–33 (2004)
Eggleston, H.: The fractional dimension of a set defined by decimal properties. Quarterly Journal of Mathematics 20, 31–36 (1949)
Hausdorff, F.: Dimension und äusseres Mass. Mathematische Annalen 79, 157–179 (1919) (English translation)
Hitchcock, J.M.: Effective Fractal Dimension Bibliography (October 2008), http://www.cs.uwyo.edu/~jhitchco/bib/dim.shtml
Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and its Applications, 2nd edn. Springer, Berlin (1997)
Lutz, J.H.: Dimension in complexity classes. SIAM Journal on Computing 32, 1236–1259 (2003)
Lutz, J.H.: The dimensions of individual strings and sequences. Information and Computation 187, 49–79 (2003)
Lutz, J.H.: A divergence formula for randomness and dimension. Technical Report cs.CC/0811.1825, Computing Research Repository (2008)
Lutz, J.H., Mayordomo, E.: Dimensions of points in self-similar fractals. SIAM Journal on Computing 38, 1080–1112 (2008)
Martin-Löf, P.: The definition of random sequences. Information and Control 9, 602–619 (1966)
Mayordomo, E.: A Kolmogorov complexity characterization of constructive Hausdorff dimension. Information Processing Letters 84(1), 1–3 (2002)
Schnorr, C.P.: A unified approach to the definition of random sequences. Mathematical Systems Theory 5, 246–258 (1971)
Schnorr, C.P.: A survey of the theory of random sequences. In: Butts, R.E., Hintikka, J. (eds.) Basic Problems in Methodology and Linguistics, pp. 193–210. D. Reidel, Dordrecht (1977)
Sullivan, D.: Entropy, Hausdorff measures old and new, and limit sets of geometrically finite Kleinian groups. Acta Mathematica 153, 259–277 (1984)
Tricot, C.: Two definitions of fractional dimension. In: Mathematical Proceedings of the Cambridge Philosophical Society, vol. 91, pp. 57–74 (1982)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lutz, J.H. (2009). A Divergence Formula for Randomness and Dimension. In: Ambos-Spies, K., Löwe, B., Merkle, W. (eds) Mathematical Theory and Computational Practice. CiE 2009. Lecture Notes in Computer Science, vol 5635. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03073-4_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-03073-4_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03072-7
Online ISBN: 978-3-642-03073-4
eBook Packages: Computer ScienceComputer Science (R0)