Robert L. Bocchino
Vikram S. Adve
ABSTRACT
While there has been extensive work on the design of software transactional memory (STM) for cache coherent shared memory systems, there has been no work on the design of an STM system for very large scale platforms containing potentially thousands of nodes. In this work, we present Cluster-STM, an STM designed for high performance on large-scale commodity clusters. Our design addresses several novel issues posed by this domain, including aggregating communication, managing locality, and distributing transactional metadata onto the nodes. We also re-evaluate several STM design choices previously studied for cache-coherent machines and conclude that, in some cases, different choices are appropriate on clusters. Finally, we show that our design scales well up to 512 processors. This is because on a cluster, the main barrier to STM scalability is the remote communication overhead imposed by the STM operations, and our design aggregates most of that communication with the communication of the underlying data.
- A.-R. Adl-Tabatabai, B. T. Lewis, V. Menon, B. R. Murphy, B. Saha, and T. Shpeisman. Compiler and runtime support for efficient software transactional memory. In ACM Conf. on Prog. Lang. Design and Implementation (PLDI), pages 26--37, New York, NY, 2006. ACM Press. Google Scholar
Digital Library
- C. Amza, A. L. Cox, S. Dwarkadas, P. Keleher, H. Lu, R. Rajamony, W. Yu, and W. Zwaenepoel. TreadMarks: Shared memory computing on networks of workstations. Computer, 29(2):18--28, 1996. Google ScholarDigital Library
- D. A. Bader and K. Madduri. Design and implementation of the HPCS graph analysis benchmark on symmetric multiprocessors. Technical report, Georgia Institute of Technology, Atlanta, GA, September 2005.Google Scholar
- P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. Addison-Wesley, 1987. Google ScholarDigital Library
- C. Blundell, J. Devietti, E. C. Lewis, and M. M. K. Martin. Making the fast case common and the uncommon case simple in unbounded transactional memory. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 24--34, New York, NY, USA, 2007. ACM Press. Google ScholarDigital Library
- D. Bonachea. Gasnet specification, v1.1. Technical Report No. UCB/CSD-02-1207, University of California at Berkeley, October 2002. Google ScholarDigital Library
- B. D. Carlstrom, A. McDonald, H. Chafi, J. Chung, C. C. Minh, C. Kozyrakis, and K. Olukotun. The Atomos transactional programming language. In ACM Conf. on Prog. Lang. Design and Implementation (PLDI), pages 1--13, New York, NY, 2006. ACM Press. Google ScholarDigital Library
- B. Chamberlain, D. Callahan, and H. Zima. Parallel programmability and the Chapel language. In Int'l. J. High Performance Comp. Applications, volume 21, pages 291--312, Thousand Oaks, CA, USA, 2007. Sage Publications, Inc. Google ScholarDigital Library
- P. Charles, C. Grothoff, V. Saraswat, C. Donawa, A. Kielstra, K. Ebcioglu, C. von Praun, and V. Sarkar. X10: An object-oriented approach to non-uniform cluster computing. In Proc. ACM SIGPLAN Conf. on Object-Oriented Prog., Systs., Langs., and Applications (OOPSLA), pages 519--538, New York, NY, 2005. ACM Press. Google ScholarDigital Library
- C. Cole and M. Herlihy. Snapshots and software transactional memory. J. Sci. Comp. Prog., 58(3):310--324, 2005. Google ScholarDigital Library
- D. Dice and N. Shavit. Understanding tradeoffs in software transactional memory. In Proc. Int'l. Symp. on Code Gen. and Optimization (CGO), pages 21--33, Washington, DC, USA, 2007. IEEE Computer Society. Google ScholarDigital Library
- T. El-Ghazawi, W. Carlson, T. Sterling, and K. Yelick. UPC: Distributed Shared-Memory Programming. Wiley-Interscience, June 2005. Google ScholarDigital Library
- K. P. Eswaran, J. N. Gray, R. A. Lorie, and I. L. Traiger. The notions of consistency and predicate locks in a database system. Commun. of the ACM, 19(11):624--633, 1976. Google ScholarDigital Library
- K. Fraser. Practical lock freedom. PhD thesis, Cambridge University Computer Laboratory, 2003.Google Scholar
- T. Harris and K. Fraser. Language support for lightweight transactions. In Proc. ACM SIGPLAN Conf. on Object-Oriented Prog., Systs., Langs., and Applications (OOPSLA), pages 388--402, New York, NY, 2003. ACM Press. Google ScholarDigital Library
- T. Harris, M. Plesko, A. Shinnar, and D. Tarditi. Optimizing memory transactions. In ACM Conf. on Prog. Lang. Design and Implementation (PLDI), pages 14--25, New York, NY, 2006. ACM Press. Google ScholarDigital Library
- M. Herlihy, V. Luchangco, M. Moir, and W. N. Scherer III. Software transactional memory for dynamic-sized data structures. In Symp. on Principles of Distrib. Comp., pages 92--101, New York, NY, 2003. ACM Press. Google ScholarDigital Library
- M. Herlihy and J. E. B. Moss. Transactional memory: Architectural support for lock-free data structures. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 289--300, New York, NY, USA, 1993. ACM Press. Google ScholarDigital Library
- M. Herlihy and Y. Sun. Distributed transactional memory for metric-space networks. In Symp. on Distrib. Comp., volume 3724. Springer, Sept. 2005. Google ScholarDigital Library
- S. Kumar, M. Chu, C. J. Hughes, P. Kundu, and A. Nguyen. Hybrid transactional memory. In ACM SIGPLAN Symp. on Principles and Practice of Parallel Prog. (PPOPP), pages 209--220, New York, NY, 2006. ACM Press. Google ScholarDigital Library
- H. T. Kung and J. T. Robinson. On optimistic methods for concurrency control. ACM Trans. on Database Systs., 6(2):213--226, 1981. Google ScholarDigital Library
- K. Manassiev, M. Mihailescu, and C. Amza. Exploiting distributed version concurrency in a transactional memory cluster. In ACM SIGPLAN Symp. on Principles and Practice of Parallel Prog. (PPOPP), pages 198--208, New York, NY, 2006. ACM Press. Google ScholarDigital Library
- V. J. Marathe, W. N. Scherer, and M. L. Scott. Design tradeoffs in modern software transactional memory systems. In Proc. Workshop on Langs., Compilers, and Run-Time Support for Scalable Systs., pages 1--7, New York, NY, 2004. ACM Press. Google ScholarDigital Library
- V. J. Marathe, W. N. Scherer III, and M. L. Scott. Adaptive software transactional memory. In Proc. Int'l. Symp. on Distrib. Comp., Cracow, Poland, Sep 2005. Earlier but expanded version available as TR 868, University of Rochester Computer Science Dept., May 2005. Google ScholarDigital Library
- M. Martin, C. Blundell, and E. Lewis. Subtleties of transactional memory atomicity semantics. IEEE Comp. Arch. Letters, 5(2), 2006. Google ScholarDigital Library
- C. C. Minh, M. Trautmann, J. Chung, A. McDonald, N. Bronson, J. Casper, C. Kozyrakis, and K. Olukotun. An effective hybrid transactional memory system with strong isolation guarantees. SIGARCH Comput. Archit. News, 35(2):69--80, 2007. Google ScholarDigital Library
- C. C. Minh, M. Trautmann, J. Chung, A. McDonald, N. Bronson, J. Casper, C. Kozyrakis, and K. Olukotun. An effective hybrid transactional memory system with strong isolation guarantees. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 69--80, New York, NY, USA, 2007. ACM Press. Google ScholarDigital Library
- R. W. Numerich and J. Reid. Co-array fortran for parallel programming. SIGPLAN Fortran Forum, 17(2):1--31, 1998. Google ScholarDigital Library
- C. H. Papadimitriou. The serializability of concurrent database updates. J. ACM, 26(4):631--653, 1979. Google ScholarDigital Library
- R. Rajwar, M. Herlihy, and K. Lai. Virtualizing transactional nemory. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 494--505, Washington, DC, 2005. IEEE Computer Society. Google ScholarDigital Library
- D. P. Reed. Implementing atomic actions on decentralized data. ACM Trans. on Comp. Systs., 1(1):3--23, 1983. Google ScholarDigital Library
- B. Saha, A.-R. Adl-Tabatabai, R. L. Hudson, C. C. Minh, and B. Hertzberg. McRT-STM: A high performance software transactional memory system for a multi-core runtime. In ACM SIGPLAN Symp. on Principles and Practice of Parallel Prog. (PPOPP), pages 187--197, New York, NY, 2006. ACM Press. Google ScholarDigital Library
- B. Saha, A.-R. Adl-Tabatabai, and Q. Jacobson. Architectural support for software transactional memory. In Proc. ACM/IEEE Int'l. Symp. on Microarch. (MICRO), pages 185--196, Washington, DC, USA, 2006. IEEE Computer Society. Google ScholarDigital Library
- N. Shavit and D. Touitou. Software transactional memory. In Symp. on Principles of Distrib. Comp., pages 204--213, New York, NY, 1995. ACM Press. Google ScholarDigital Library
- T. Shpeisman, V. Menon, A.-R. Adl-Tabatabai, S. Balensiefer, D. Grossman, R. L. Hudson, K. F. Moore, and B. Saha. Enforcing isolation and ordering in STM. ACM Conf. on Prog. Lang. Design and Implementation (PLDI), 42(6):78--88, 2007. Google ScholarDigital Library
- A. Shriraman, M. F. Spear, H. Hossain, V. J. Marathe, S. Dwarkadas, and M. L. Scott. An integrated hardware-software approach to flexible transactional memory. In Proc. Int'l. Conf. on Comp. Arch. (ISCA), pages 104--115, New York, NY, USA, 2007. ACM Press. Google ScholarDigital Library
- Sun Microsystems, Inc. The Fortress language specification, version 1.0 β. Technical report, Sun Microsystems, Inc., March 2007.Google Scholar
- K. Yelick, L. Semenzato, G. Pike, C. Miyamoto, B. Liblit, A. Krishnamurthy, P. Hilfinger, S. Graham, D. Gay, P. Colella, and A. Aiken. Titanium: A high-performance Java dialect. Concurrency -- Practice and Experience, 10(11-13):825--836, Sept-Nov 1998.Google Scholar
Index Terms
- Software transactional memory for large scale clusters
Recommendations
Lowering Conflicts of High Contention Software Transactional Memory
CSSE '08: Proceedings of the 2008 International Conference on Computer Science and Software Engineering - Volume 03Two concurrent transactions are said to conflict based on linearizability semantics if they access the same shared data and at least one of them modifies that data. In many applications enforcing the strict linearizability semantics over the entire read-...
Software Distributed Shared Memory with Transactional Coherence - A Software Engine to Run Transactional Shared-memory Parallel Applications on Clusters
PDP '10: Proceedings of the 2010 18th Euromicro Conference on Parallel, Distributed and Network-based ProcessingTransactional Memory is a novel, promising approach for simplifying parallel programming and increasing its acceptance and diffusion. Until now, almost all the research work on TM has been focused on shared-memory architectures, while very limited ...
Software transactional memory for multicore embedded systems
LCTES '09Embedded systems, like general-purpose systems, can benefit from parallel execution on a symmetric multicore platform. Unfortunately, concurrency issues present in general-purpose programming also apply to embedded systems, protection from which is ...
Comments