skip to main content
10.1145/2882903.2915227acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

T-Part: Partitioning of Transactions for Forward-Pushing in Deterministic Database Systems

Authors Info & Claims
Published:26 June 2016Publication History

ABSTRACT

Deterministic database systems have been shown to yield high throughput on a cluster of commodity machines while ensuring the strong consistency between replicas, provided that the data can be well-partitioned on these machines. However, data partitioning can be suboptimal for many reasons in real-world applications. In this paper, we present T-Part, a transaction execution engine that partitions transactions in a deterministic database system to deal with the unforeseeable workloads or workloads whose data are hard to partition. By modeling the dependency between transactions as a T-graph and continuously partitioning that graph, T-Part allows each transaction to know which later transactions on other machines will read its writes so that it can push forward the writes to those later transactions immediately after committing. This forward-pushing reduces the chance that the later transactions stall due to the unavailability of remote data. We implement a prototype for T-Part. Extensive experiments are conducted and the results demonstrate the effectiveness of T-Part.

References

  1. Calvin source code. https://github.com/yaledb/calvin.Google ScholarGoogle Scholar
  2. Elasql. http://www.elasql.org.Google ScholarGoogle Scholar
  3. Nuodb. http://www.nuodb.com.Google ScholarGoogle Scholar
  4. The top 5 aws ec2 performance problems. http://www.datadoghq.com/wp-content/uploads/2015/06/Top-5-AWS-Ec2-Performance-Problems-Guide-Ebook.pdf.Google ScholarGoogle Scholar
  5. Vanilladb. http://www.vanilladb.org.Google ScholarGoogle Scholar
  6. C. Amza, A. L. Cox, and W. Zwaenepoel. Conflict-aware scheduling for dynamic content applications. In USENIX Symposium on Internet Technologies and Systems, volume 21, page 22, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. T. Cao, M. Vaz Salles, B. Sowell, Y. Yue, A. Demers, J. Gehrke, and W. White. Fast checkpoint recovery algorithms for frequently consistent applications. In Proc. of SIGMOD, pages 265--276. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. T. Chandra, R. Griesemer, and J. Redstone. Paxos made live-an engineering perspective (2006 invited talk). In Proc. of PODC, volume 7, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. C. Curino, E. Jones, Y. Zhang, and S. Madden. Schism: a workload-driven approach to database replication and partitioning. Proc. of the VLDB Endowment, 3(1--2):48--57, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Das, D. Agrawal, and A. El Abbadi. G-store: A scalable data store for transactional multi key access in the cloud. In Proceedings of the 1st ACM Symposium on Cloud Computing, SoCC '10, pages 163--174, New York, NY, USA, 2010. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. Elnikety, S. Dropsho, and W. Zwaenepoel. Tashkent+: Memory-aware load balancing and update filtering in replicated databases. ACM SIGOPS Operating Systems Review, 41(3):399--412, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. J. M. Faleiro and D. J. Abadi. Rethinking serializable multiversion concurrency control. Proc. of the VLDB Endowment, 8(11):1190--1201, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. J. M. Faleiro, A. Thomson, and D. J. Abadi. Lazy evaluation of transactions in database systems. In Proc. of SIGMOD, pages 15--26. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. P. Hunt, M. Konar, F. P. Junqueira, and B. Reed. Zookeeper: wait-free coordination for internet-scale systems. In Proc. of the 2010 USENIX conference on USENIX annual technical conference, volume 8, pages 11--11, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. R. Kallman, H. Kimura, J. Natkins, A. Pavlo, A. Rasin, S. Zdonik, E. P. Jones, S. Madden, M. Stonebraker, Y. Zhang, et al. H-store: a high-performance, distributed main memory transaction processing system. Proc. of the VLDB Endowment, 1(2):1496--1499, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. G. Karypis and V. Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM journal on Scientific Computing, 20(1):359--392, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. B. Kemme and G. Alonso. Don't be lazy, be consistent: Postgres-r, a new way to implement database replication. In Proc. of VLDB, pages 134--143, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. E. B. Nightingale, P. M. Chen, and J. Flinn. Speculative execution in a distributed file system. ACM Transactions on Computer Systems (TOCS), 24(4):361--392, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. V. S. Pai, M. Aron, G. Banga, M. Svendsen, P. Druschel, W. Zwaenepoel, and E. Nahum. Locality-aware request distribution in cluster-based network servers. In ACM Sigplan Notices, volume 33, pages 205--216. ACM, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Pavlo, C. Curino, and S. Zdonik. Skew-aware automatic database partitioning in shared-nothing, parallel oltp systems. In Proc. of SIGMOD, pages 61--72. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. A. Quamar, K. A. Kumar, and A. Deshpande. Sword: scalable workload-aware data placement for transactional workloads. In Proc. of EDBT, pages 430--441. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. B. Reed and F. P. Junqueira. A simple totally ordered broadcast protocol. In Proc. of the 2nd Workshop on Large-Scale Distributed Systems and Middleware, page 2. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. C. Sapia. Promise: Predicting query behavior to enable predictive caching strategies for olap systems. In Proc. of Data Warehousing and Knowledge Discovery, pages 224--233. Springer, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. A. J. Smith. Sequentiality and prefetching in database systems. ACM Transactions on Database Systems (TODS), 3(3):223--247, 1978. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. G. Soundararajan, M. Mihailescu, and C. Amza. Context-aware prefetching at the storage server. In Proc. of the USENIX Annual Technical Conference, pages 377--390, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. I. Stanton and G. Kliot. Streaming graph partitioning for large distributed graphs. In Proc. of SIGKDD, pages 1222--1230. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. M. Stonebraker, S. Madden, D. J. Abadi, S. Harizopoulos, N. Hachem, and P. Helland. The end of an architectural era:(it's time for a complete rewrite). In Proc. of the VLDB Endowment, pages 1150--1160. VLDB Endowment, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. F. Tauheed, T. Heinis, F. Schürmann, H. Markram, and A. Ailamaki. Scout: Prefetching for latent structure following queries. Proc. of the VLDB Endowment, 5(11):1531--1542, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. A. Thomson and D. J. Abadi. The case for determinism in database systems. Proc. of the VLDB Endowment, 3(1--2):70--80, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. A. Thomson, T. Diamond, S.-C. Weng, K. Ren, P. Shao, and D. J. Abadi. Calvin: Fast distributed transactions for partitioned database systems. In Proc. of SIGMOD, pages 1--12. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. E.-J. van Baaren. Wikibench: A distributed, wikipedia based web application benchmark. Master's thesis, VU University Amsterdam, 2009.Google ScholarGoogle Scholar
  32. T. Yamamuro, Y. Suga, N. Kotani, T. Hitaka, and M. Yamamuro. Buffer cache de-duplication for query dispatch in replicated databases. In Database Systems for Advanced Applications, pages 352--366. Springer, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. S. Yang, X. Yan, B. Zong, and A. Khan. Towards effective partition management for large graphs. In Proc. of SIGMOD, pages 517--528. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. G.-W. You, S.-W. Hwang, and N. Jain. Ursa: Scalable load and power management in cloud storage systems. ACM Transactions on Storage, 9(1):1, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. X. Zhang, M. Barrientos, J. B. Chen, and M. Seltzer. Hacc: An architecture for cluster-based web servers. In Proc. of the USENIX Windows NT Symposium, volume 3, pages 16--16. USENIX Association, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. V. Zuikeviciute and F. Pedone. Conflict-aware load-balancing techniques for database replication. In Proc. of ACM SAC, pages 2169--2173. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. T-Part: Partitioning of Transactions for Forward-Pushing in Deterministic Database Systems

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        SIGMOD '16: Proceedings of the 2016 International Conference on Management of Data
        June 2016
        2300 pages
        ISBN:9781450335317
        DOI:10.1145/2882903

        Copyright © 2016 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 26 June 2016

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        Overall Acceptance Rate785of4,003submissions,20%

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader