ABSTRACT
The Erlang Term Storage (ETS) is an important component of the Erlang runtime system, especially when parallelism enters the picture, as it provides an area where processes can share data. It is therefore important that ETS's implementation is efficient, flexible, but also as scalable as possible. In this paper we document and describe the current implementation of ETS in detail, discuss the main data structures that support it, and present the main points of its evolution across Erlang/OTP releases. More importantly, we measure the scalability of its implementations, the effects of its tuning options, identify bottlenecks, and suggest changes and alternative designs that can improve both its performance and its scalability.
- G. Adelson-Velskii and E. M. Landis. An algorithm for the organization of information. In Proceedings of the USSR Academy of Sciences, volume 146, pages 263--266, 1962. In Russian.Google Scholar
- Stavros Aronis, Nikolaos Papaspyrou, Katerina Roukounaki, Konstantinos Sagonas, Yiannis Tsiouris, and Ioannis E. Venetis. A scalability benchmark suite for Erlang/OTP. In Proceedings of the Eleventh ACM SIGPLAN Workshop on Erlang, pages 33--42. ACM, 2012. Google ScholarDigital Library
- Michael A. Bender, Jeremy T. Fineman, Seth Gilbert, and Bradley C. Kuszmaul. Concurrent cache-oblivious B-trees. In Proceedings of the Seventeenth Annual ACM Symposium on Parallelism in Algorithms and Architectures, pages 228--237. ACM, 2005. Google ScholarDigital Library
- Nathan G. Bronson, Jared Casper, Hassan Chafi, and Kunle Olukotun. A practical concurrent binary search tree. In Proceedings of the 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 257--268. ACM, 2010. Google ScholarDigital Library
- Keir Fraser. Practical lock-freedom. PhD thesis, University of Cambridge Computer Laboratory, 2004.Google Scholar
- Scott Lystig Fritchie. A study of Erlang ETS table implementations and performance. In Proceedings of the 2003 ACM SIGPLAN Workshop on Erlang, pages 43--55. ACM, 2003. Google ScholarDigital Library
- Thomas E. Hart, Paul E. McKenney, Angela Demke Brown, and Jonathan Walpole. Performance of memory reclamation for lockless synchronization. Journal of Parallel and Distributed Computing, 67(12):1270--1285, 2007. Google ScholarDigital Library
- Maurice Herlihy, Yossi Lev, Victor Luchangco, and Nir Shavit. A simple optimistic skiplist algorithm. In Structural Information and Communication Complexity, pages 124--138. Springer, 2007. Google ScholarDigital Library
- Maurice Herlihy and Nir Shavit. The Art of Multiprocessor Programming, Revised Reprint. Morgan Kaufmann, 2012. Google ScholarDigital Library
- Per-Åke Larson. Linear hashing with partial expansions. In Proceedings of the Sixth International Conference on Very Large Data Bases, pages 224--232. VLDB Endowment, 1980. Google ScholarDigital Library
- Maged M. Michael. High performance dynamic lock-free hash tables and list-based sets. In Proceedings of the Fourteenth Annual ACM Symposium on Parallel Algorithms and Architectures, pages 73--82. ACM, 2002. Google ScholarDigital Library
- Maged M. Michael. Hazard pointers: Safe memory reclamation for lock-free objects. IEEE Transactions on Parallel and Distributed Systems, 15(6):491--504, 2004. Google ScholarDigital Library
- Bradford Nichols, Dick Buttlar, and Jacqueline Proulx Farrell. Pthreads programming. O'Reilly & Associates, Inc., Sebastopol, CA, USA, 1996. Google ScholarDigital Library
- Patrik Nyblom. Erlang ETS tables and software transactional memory: how transactions make ETS tables more like ordinary actors. In Proceedings of the Tenth ACM SIGPLAN Workshop on Erlang, pages 2--13. ACM, 2011. Google ScholarDigital Library
- Mark S. Papamarcos and Janak H. Patel. A low-overhead coherence solution for multiprocessors with private cache memories. In Proceedings of the 11th Annual Symposium on Computer Architecture, pages 348--354. ACM, 1984. Google ScholarDigital Library
- Ori Shalev and Nir Shavit. Split-ordered lists: Lock-free extensible hash tables. J. ACM, 53(3):379--405, May 2006. See also expanded version available at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.103.4226&rep=rep1&type=pdf (accessed 9 June 2013). Google ScholarDigital Library
- Nir Shavit and Dan Touitou. Software transactional memory. Distributed Computing, 10(2):99--116, 1997.Google ScholarCross Ref
Index Terms
On the scalability of the Erlang term storage
Recommendations
More scalable ordered set for ETS using adaptation
Erlang '14: Proceedings of the Thirteenth ACM SIGPLAN workshop on ErlangThe Erlang Term Storage (ETS) is a key component of the runtime system and standard library of Erlang/OTP. In particular, on big multicores, the performance of many applications that use ETS as a shared key-value store heavily depends on the scalability ...
Scalable persistent storage for Erlang: theory and practice
Erlang '13: Proceedings of the twelfth ACM SIGPLAN workshop on ErlangThe many core revolution makes scalability a key property. The RELEASE project aims to improve the scalability of Erlang on emergent commodity architectures with 100,000 cores. Such architectures require scalable and available persistent storage on up ...
Improving the network scalability of Erlang
As the number of cores grows in commodity architectures so does the likelihood of failures. A distributed actor model potentially facilitates the development of reliable and scalable software on these architectures. Key components include lightweight ...
Comments