skip to main content
10.1145/2505305.2505308acmconferencesArticle/Chapter ViewAbstractPublication PagesicfpConference Proceedingsconference-collections
research-article

On the scalability of the Erlang term storage

Published:28 September 2013Publication History

ABSTRACT

The Erlang Term Storage (ETS) is an important component of the Erlang runtime system, especially when parallelism enters the picture, as it provides an area where processes can share data. It is therefore important that ETS's implementation is efficient, flexible, but also as scalable as possible. In this paper we document and describe the current implementation of ETS in detail, discuss the main data structures that support it, and present the main points of its evolution across Erlang/OTP releases. More importantly, we measure the scalability of its implementations, the effects of its tuning options, identify bottlenecks, and suggest changes and alternative designs that can improve both its performance and its scalability.

References

  1. G. Adelson-Velskii and E. M. Landis. An algorithm for the organization of information. In Proceedings of the USSR Academy of Sciences, volume 146, pages 263--266, 1962. In Russian.Google ScholarGoogle Scholar
  2. Stavros Aronis, Nikolaos Papaspyrou, Katerina Roukounaki, Konstantinos Sagonas, Yiannis Tsiouris, and Ioannis E. Venetis. A scalability benchmark suite for Erlang/OTP. In Proceedings of the Eleventh ACM SIGPLAN Workshop on Erlang, pages 33--42. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Michael A. Bender, Jeremy T. Fineman, Seth Gilbert, and Bradley C. Kuszmaul. Concurrent cache-oblivious B-trees. In Proceedings of the Seventeenth Annual ACM Symposium on Parallelism in Algorithms and Architectures, pages 228--237. ACM, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Nathan G. Bronson, Jared Casper, Hassan Chafi, and Kunle Olukotun. A practical concurrent binary search tree. In Proceedings of the 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 257--268. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Keir Fraser. Practical lock-freedom. PhD thesis, University of Cambridge Computer Laboratory, 2004.Google ScholarGoogle Scholar
  6. Scott Lystig Fritchie. A study of Erlang ETS table implementations and performance. In Proceedings of the 2003 ACM SIGPLAN Workshop on Erlang, pages 43--55. ACM, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Thomas E. Hart, Paul E. McKenney, Angela Demke Brown, and Jonathan Walpole. Performance of memory reclamation for lockless synchronization. Journal of Parallel and Distributed Computing, 67(12):1270--1285, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Maurice Herlihy, Yossi Lev, Victor Luchangco, and Nir Shavit. A simple optimistic skiplist algorithm. In Structural Information and Communication Complexity, pages 124--138. Springer, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Maurice Herlihy and Nir Shavit. The Art of Multiprocessor Programming, Revised Reprint. Morgan Kaufmann, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Per-Åke Larson. Linear hashing with partial expansions. In Proceedings of the Sixth International Conference on Very Large Data Bases, pages 224--232. VLDB Endowment, 1980. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Maged M. Michael. High performance dynamic lock-free hash tables and list-based sets. In Proceedings of the Fourteenth Annual ACM Symposium on Parallel Algorithms and Architectures, pages 73--82. ACM, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Maged M. Michael. Hazard pointers: Safe memory reclamation for lock-free objects. IEEE Transactions on Parallel and Distributed Systems, 15(6):491--504, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Bradford Nichols, Dick Buttlar, and Jacqueline Proulx Farrell. Pthreads programming. O'Reilly & Associates, Inc., Sebastopol, CA, USA, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Patrik Nyblom. Erlang ETS tables and software transactional memory: how transactions make ETS tables more like ordinary actors. In Proceedings of the Tenth ACM SIGPLAN Workshop on Erlang, pages 2--13. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mark S. Papamarcos and Janak H. Patel. A low-overhead coherence solution for multiprocessors with private cache memories. In Proceedings of the 11th Annual Symposium on Computer Architecture, pages 348--354. ACM, 1984. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Ori Shalev and Nir Shavit. Split-ordered lists: Lock-free extensible hash tables. J. ACM, 53(3):379--405, May 2006. See also expanded version available at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.103.4226&rep=rep1&type=pdf (accessed 9 June 2013). Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Nir Shavit and Dan Touitou. Software transactional memory. Distributed Computing, 10(2):99--116, 1997.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. On the scalability of the Erlang term storage

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in
            • Published in

              cover image ACM Conferences
              Erlang '13: Proceedings of the twelfth ACM SIGPLAN workshop on Erlang
              September 2013
              100 pages
              ISBN:9781450323857
              DOI:10.1145/2505305

              Copyright © 2013 ACM

              Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 28 September 2013

              Permissions

              Request permissions about this article.

              Request Permissions

              Check for updates

              Qualifiers

              • research-article

              Acceptance Rates

              Erlang '13 Paper Acceptance Rate12of15submissions,80%Overall Acceptance Rate51of68submissions,75%

              Upcoming Conference

              ICFP '24

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader