Skip to main content

A Simpler Analysis of Burrows-Wheeler Based Compression

  • Conference paper
Combinatorial Pattern Matching (CPM 2006)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4009))

Included in the following conference series:

Abstract

In this paper we present a new technique for worst-case analysis of compression algorithms which are based on the Burrows-Wheeler Transform. We deal mainly with the algorithm purposed by Burrows and Wheeler in their first paper on the subject [6], called bw0. This algorithm consists of the following three steps: 1) Compute the Burrows-Wheeler transform of the text, 2) Convert the transform into a sequence of integers using the move-to-front algorithm, 3) Encode the integers using Arithmetic code or any order-0 encoding (possibly with run-length encoding).

We prove a strong upper bound on the worst-case compression ratio of this algorithm. This bound is significantly better than bounds known to date and is obtained via simple analytical techniques. Specifically, we show that for any input string s, and μ> 1, the length of the compressed string is bounded by μ |s| H k (s) + log(ζ(μ)) |s| + g k where H k is the k-th order empirical entropy, g k is a constant depending only on k and on the size of the alphabet, and \(\zeta(\mu) = \frac{1}{1^\mu} + \frac{1}{2^\mu} + \ldots \) is the standard zeta function. As part of the analysis we prove a result on the compressibility of integer sequences, which is of independent interest.

Finally, we apply our techniques to prove a worst-case bound on the compression ratio of a compression algorithm based on the Burrows-Wheeler transform followed by distance coding, for which worst-case guarantees have never been given. We prove that the length of the compressed string is bounded by 1.7286 |s| H k (s) + g k . This bound is better than the bound we give for bw0.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. The canterbury corpus, http://corpus.canterbury.ac.nz

  2. Abel, J.: Web page about Distance Coding, http://www.data-compression.info/Algorithms/DC/

  3. Apostolico, A., Fraenkel, A.S.: Robust transmission of unbounded strings using fibonacci representations. IEEE Transactions on Information Theory 33(2), 238–245 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bentley, J.L., Sleator, D.D., Tarjan, R.E., Wei, V.K.: A locally adaptive data compression scheme. Communications of the ACM 29(4), 320–330 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  5. Binder, E.: Distance coder. Usenet group comp.compression (2000)

    Google Scholar 

  6. Burrows, M., Wheeler, D.J.: A block sorting lossless data compression algorithm. Technical Report 124, Digital Equipment Corporation, Palo Alto, California (1994)

    Google Scholar 

  7. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, ch. 16.3, 2nd edn., pp. 385–392. MIT Press and McGraw-Hill (2001)

    Google Scholar 

  8. Deorowicz, S.: Second step algorithms in the Burrows–Wheeler compression algorithm. Software – Practice and Experience 32(2), 99–111 (2002)

    Article  MATH  Google Scholar 

  9. Elias, P.: Universal codeword sets and representation of the integers. IEEE Trans. on Information Theory 21(2), 194–203 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  10. Ferragina, P., Giancarlo, R., Manzini, G., Sciortino, M.: Boosting textual compression in optimal linear time. Journal of the ACM 52, 688–713 (2005)

    Article  MathSciNet  Google Scholar 

  11. Fraenkel, A.S., Klein, S.T.: Robust universal complete codes for transmission and compression. Discrete Applied Mathematics 64(1), 31–55 (1996)

    Article  MATH  Google Scholar 

  12. Huffman, D.A.: A method for the construction of minimum-redundancy codes. Proceedings of the IRE 40(9), 1098–1101 (1952)

    Article  Google Scholar 

  13. Lelewer, D.A., Hirschberg, D.S.: Data compression. ACM Computing Surveys 19(3), 261–296 (1987)

    Article  MATH  Google Scholar 

  14. Manzini, G.: Personal communication

    Google Scholar 

  15. Manzini, G.: An analysis of the Burrows-Wheeler transform. Journal of the ACM 48(3), 407–430 (2001)

    Article  MathSciNet  Google Scholar 

  16. Manzini, G., Ferragina, P.: Engineering a lightweight suffix array construction algorithm. Algorithmica 40, 33–50 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  17. Moffat, A., Neal, R.M., Witten, I.H.: Arithmetic coding revisited. ACM Trans. Inf. Syst. 16(3), 256–294 (1998)

    Article  Google Scholar 

  18. Seward, J.: bzip2, a program and library for data compression, http://www.bzip.org/

  19. Witten, I.H., Neal, R.M., Cleary, J.G.: Arithmetic coding for data compression. Communications of the ACM 30(6), 520–540 (1987)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kaplan, H., Landau, S., Verbin, E. (2006). A Simpler Analysis of Burrows-Wheeler Based Compression. In: Lewenstein, M., Valiente, G. (eds) Combinatorial Pattern Matching. CPM 2006. Lecture Notes in Computer Science, vol 4009. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11780441_26

Download citation

  • DOI: https://doi.org/10.1007/11780441_26

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-35455-0

  • Online ISBN: 978-3-540-35461-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics