skip to main content
research-article

Algorithm 938: Compressing circular buffers

Published: 05 March 2014 Publication History

Abstract

Data sequences generated by on-line sensors can become arbitrarily large and must, therefore, be pared down to fit into available memory. For situations where only the most recent data is of interest, this problem can be solved with optimal efficiency by a simple circular buffer: it fills each memory location with useful data, and requires just one write to memory per update. The algorithm presented here provides essentially the same efficiency, but while maintaining a continuously updated, fixed-size, compressed representation of the entire data sequence. Each value in these compressed sequences represents a statistic (an average, maximum, random sample, etc.) computed over a contiguous chunk of the original sequence. Compressing circular buffers gain their efficiency by using an alternative indexing sequence, based on well-known principles of elementary number theory, to ensure that each newly written value gets stored in the unoccupied location created when the two oldest sequential over-sampled values are compressed into one. The associated Java implementation supports a variety of aggregating statistics and is used to compare the algorithm's performance with a more obvious approach (doubling).

Supplementary Material

ZIP File (938.zip)
Software for Compressing circular buffers

References

[1]
R. C. Baker and G. Harman. 2001. The difference between consecutive primes, II. Proc. Lond. Math. Soc. 3, 83, 532--562.
[2]
Tony F. Chan, Gene H. Golub, and Randal J. LeVeque. 1983. Algorithms for computing the sample variance: Analysis and recommendations. Amer. Stat. 37, 3, 242--247.
[3]
Charles Vanden Eynden. 1970. Number Theory: An Introduction to Proof. International Textbook Company, Scranton, Pa.
[4]
Deepak Ganesan, Ben Greenstein, Denis Perelyubskiy, Deborah Estrin, and John Heidemann. 2003. An evaluation of multi-resolution storage for sensor networks. In Proceedings of the 1st International Conference on Embedded Networked Sensor Systems. ACM, New York, 89--102.
[5]
Solomon W. Golomb. 1982. Shift Register Sequences. Aegean Park Press, Walnut Creek, CA.
[6]
Ben Greenstein, Christopher Mar, Alex Pesterev, Shahin Farshchi, Eddie Kohler, Jack Judy, and Deborah Estrin. 2006. Capturing high-frequency phenomena using a bandwidth-limited sensor network. In Proceedings of the 4th International Conference on Embedded Networked Sensor Systems. ACM, New York, 279--292.
[7]
Donald E. Knuth. 1997. The Art of Computer Programming, Volume 1: Fundamental Algorithms. 3rd Ed. Addison-Wesley, Reading, MA.
[8]
Yi-Ting Lin, Wen-Chi Shiue, and Ing-Jer Huang. 2008. A multi-resolution AHB bus tracer for real-time compression in a circular buffer. In Proceedings of the 45th Annual Design Automation Conference. IEEE, New York, 862--865.
[9]
Ivan Niven and Herbert S. Zuckerman. 1966. An Introduction to the Theory of Numbers. Wiley, New York.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Mathematical Software
ACM Transactions on Mathematical Software  Volume 40, Issue 2
February 2014
161 pages
ISSN:0098-3500
EISSN:1557-7295
DOI:10.1145/2594412
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 March 2014
Accepted: 01 September 2013
Revised: 01 March 2011
Received: 01 April 2010
Published in TOMS Volume 40, Issue 2

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. Average
  2. complete residue system
  3. compressing circular buffer
  4. data aggregation
  5. data sequence
  6. doubling
  7. dynamically sized arrays
  8. lossy compression
  9. maximum
  10. minimum
  11. modular arithmetic
  12. number theory
  13. random sample
  14. relatively prime
  15. sum

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 407
    Total Downloads
  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)4
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media