skip to main content
10.1145/3183713.3196912acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

Carousel: Low-Latency Transaction Processing for Globally-Distributed Data

Published: 27 May 2018 Publication History

Abstract

The trend towards global applications and services has created an increasing demand for transaction processing on globally-distributed data. Many database systems, such as Spanner and CockroachDB, support distributed transactions but require a large number of wide-area network roundtrips to commit each transaction and ensure the transaction's state is durably replicated across multiple datacenters. This can significantly increase transaction completion time, resulting in developers replacing database-level transactions with their own error-prone application-level solutions.
This paper introduces Carousel, a distributed database system that provides low-latency transaction processing for multi-partition globally-distributed transactions. Carousel shortens transaction processing time by reducing the number of sequential wide-area network round trips required to commit a transaction and replicate its results while maintaining serializability. This is possible in part by using information about a transaction's potential write set to enable transaction processing, including any necessary remote read operations, to overlap with 2PC and state replication. Carousel further reduces transaction completion time by introducing a consensus protocol that can perform state replication in parallel with 2PC. For a multi-partition 2-round Fixed-set Interactive (2FI) transaction, Carousel requires at most two wide-area network roundtrips to commit the transaction when there are no failures, and only one round trip in the common case if local replicas are available.

References

[1]
Atul Adya, Robert Gruber, Barbara Liskov, and Umesh Maheshwari. 1995. Efficient Optimistic Concurrency Control Using Loosely Synchronized Clocks SIGMOD.
[2]
Divy Agrawal, Amr El Abbadi, and Kenneth Salem. 2015. A Taxonomy of Partitioned Replicated Cloud-based Database Systems. IEEE Data Eng. Bull. Vol. 38, 1 (2015).
[3]
Marcos K. Aguilera, Arif Merchant, Mehul Shah, Alistair Veitch, and Christos Karamanolis. 2007. Sinfonia: A New Paradigm for Building Scalable Distributed Systems SOSP.
[4]
Jason Baker, Chris Bond, James C. Corbett, JJ Furman, Andrey Khorlin, James Larson, Jean-Michel Leon, Yawei Li, Alexander Lloyd, and Vadim Yushprakh. 2011. Megastore: Providing Scalable, Highly Available Storage for Interactive Services Proceedings of the Conference on Innovative Data system Research (CIDR).
[5]
Philip A. Bernstein, Istvan Cseri, Nishant Dani, Nigel Ellis, Ajay Kalhan, Gopal Kakivaya, David B. Lomet, Ramesh Manne, Lev Novik, and Tomas Talius. 2011. Adapting microsoft SQL server for cloud computing ICDE.
[6]
Nathan Bronson, Zach Amsden, George Cabrera, Prasad Chakka, Peter Dimov, Hui Ding, Jack Ferris, Anthony Giardullo, Sachin Kulkarni, Harry Li, Mark Marchukov, Dmitri Petrov, Lovro Puzar, Yee Jiun Song, and Venkat Venkataramani. 2013. TAO: Facebook's Distributed Data Store for the Social Graph USENIX ATC.
[7]
Mike Burrows. 2006. The Chubby Lock Service for Loosely-coupled Distributed Systems OSDI.
[8]
Cockroach Labs. 2017. CockroachDB. https://github.com/cockroachdb/cockroach. (2017).
[9]
Brian F. Cooper, Raghu Ramakrishnan, Utkarsh Srivastava, Adam Silberstein, Philip Bohannon, Hans-Arno Jacobsen, Nick Puz, Daniel Weaver, and Ramana Yerneni. 2008. PNUTS: Yahoo!'s Hosted Data Serving Platform. VLDB (2008).
[10]
Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In SoCC.
[11]
James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, J. J. Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, David Nagle, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Yasushi Saito, Michal Szymaniak, Christopher Taylor, Ruth Wang, and Dale Woodford. 2012. Spanner: Google's Globally-distributed Database. In OSDI.
[12]
CoreOS. 2017. Raft Implementation. https://github.com/coreos/etcd/tree/master. (2017).
[13]
James Cowling and Barbara Liskov. 2012. Granola: Low-overhead Distributed Transaction Coordination USENIX ATC.
[14]
Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, and Werner Vogels. 2007. Dynamo: Amazon's Highly Available Key-value Store. In SOSP.
[15]
Akon Dey, Alan Fekete, Raghunath Nambiar, and Uwe Rohm. 2014. YCSB T: Benchmarking web-scale transactional databases ICDEW.
[16]
Robert Escriva and Robbert van Renesse. 2016. Consus: Taming the Paxi. CoRR Vol. abs/1612.03457 (2016).
[17]
Google. 2017. gRPC-go. https://github.com/grpc/grpc-go. (2017).
[18]
Stephen Hemminger. 2005. Network Emulation with NetEm. In Australia's 6th National Linux Conference.
[19]
Patrick Hunt, Mahadev Konar, Flavio P. Junqueira, and Benjamin Reed. 2010. ZooKeeper: Wait-free Coordination for Internet-scale Systems USENIX ATC.
[20]
Flavio Paiva Junqueira, Benjamin C. Reed, and Marco Serafini. 2011. Zab: High-performance broadcast for primary-backup systems Proceedings of the 2011 IEEE/IFIP International Conference on Dependable Systems and Networks.
[21]
Robert Kallman, Hideaki Kimura, Jonathan Natkins, Andrew Pavlo, Alexander Rasin, Stanley Zdonik, Evan P. C. Jones, Samuel Madden, Michael Stonebraker, Yang Zhang, John Hugg, and Daniel J. Abadi. 2008. H-store: A High-performance, Distributed Main Memory Transaction Processing System. VLDB.
[22]
David Karger, Eric Lehman, Tom Leighton, Rina Panigrahy, Matthew Levine, and Daniel Lewin. 1997. Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web. In Proceedings of the Twenty-ninth Annual ACM Symposium on Theory of Computing. ACM.
[23]
Tim Kraska, Gene Pang, Michael J. Franklin, Samuel Madden, and Alan Fekete. 2013. MDCC: Multi-data Center Consistency. In EuroSys.
[24]
Avinash Lakshman and Prashant Malik. 2010. Cassandra: A Decentralized Structured Storage System. SIGOPS Oper. Syst. Rev. Vol. 44, 2 (2010).
[25]
Leslie Lamport. 1998. The Part-time Parliament. ACM Trans. Comput. Syst. Vol. 16, 2 (1998).
[26]
Leslie Lamport. 2001. Paxos Made Simple. Technical Report, Microsoft (2001).
[27]
Leslie Lamport. 2005. Generalized Consensus and Paxos. Technical Report, Microsoft (2005).
[28]
Leslie Lamport. 2006. Fast Paxos. Distributed Computing Vol. 19 (October . 2006).
[29]
Leslie Lamport and Mike Massa. 2004. Cheap Paxos. Technical Report, Microsoft (2004).
[30]
Costin Leau. 2013. Spring Data Redis - Retwis-J. https://docs.spring.io/spring-data/data-keyvalue/examples/retwisj/current/. (2013).
[31]
Barbara Liskov, Miguel Castro, Liuba Shrira, and Atul Adya. 1999. Providing Persistent Objects in Distributed Systems ECOOP.
[32]
Wyatt Lloyd, Michael J. Freedman, Michael Kaminsky, and David G. Andersen. 2011. Don'T Settle for Eventual: Scalable Causal Consistency for Wide-area Storage with COPS. In SOSP.
[33]
Wyatt Lloyd, Michael J. Freedman, Michael Kaminsky, and David G. Andersen. 2013. Stronger Semantics for Low-latency Geo-replicated Storage NSDI.
[34]
Hatem Mahmoud, Faisal Nawab, Alexander Pucher, Divyakant Agrawal, and Amr El Abbadi. 2013. Low-latency Multi-datacenter Databases Using Replicated Commit. VLDB.
[35]
Yanhua Mao, Flavio P. Junqueira, and Keith Marzullo. 2008. Mencius: Building Efficient Replicated State Machines for WANs OSDI.
[36]
Iulian Moraru, David G. Andersen, and Michael Kaminsky. 2013. There is More Consensus in Egalitarian Parliaments SOSP.
[37]
Shuai Mu, Yang Cui, Yang Zhang, Wyatt Lloyd, and Jinyang Li. 2014. Extracting More Concurrency from Distributed Transactions OSDI.
[38]
Shuai Mu, Lamont Nelson, Wyatt Lloyd, and Jinyang Li. 2016. Consolidating Concurrency Control and Consensus for Commits under Conflicts OSDI.
[39]
Brian M. Oki and Barbara H. Liskov. 1988. Viewstamped Replication: A New Primary Copy Method to Support Highly-Available Distributed Systems. In PODC.
[40]
Diego Ongaro and John Ousterhout. 2014. In Search of an Understandable Consensus Algorithm USENIX ATC.
[41]
Andrew Pavlo. 2017. What Are We Doing With Our Lives?: Nobody Cares About Our Concurrency Control Research. In SIGMOD.
[42]
Dan R. K. Ports, Jialin Li, Vincent Liu, Naveen Kr. Sharma, and Arvind Krishnamurthy. 2015. Designing Distributed Systems Using Approximate Synchrony in Data Center Networks NSDI.
[43]
Yair Sovran, Russell Power, Marcos K. Aguilera, and Jinyang Li. 2011. Transactional Storage for Geo-replicated Systems. In SOSP.
[44]
D. B. Terry, M. M. Theimer, Karin Petersen, A. J. Demers, M. J. Spreitzer, and C. H. Hauser. 1995. Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System SOSP.
[45]
Alexander Thomson and Daniel J. Abadi. 2010. The Case for Determinism in Database Systems. Proc. VLDB Endowment Vol. 3, 1--2 (2010), 70--80.
[46]
Alexander Thomson, Thaddeus Diamond, Shu-Chun Weng, Kun Ren, Philip Shao, and Daniel J. Abadi. 2012. Calvin: Fast Distributed Transactions for Partitioned Database Systems SIGMOD.
[47]
UWSysLab. 2017. TAPIR Implementation. https://github.com/UWSysLab/tapir. (2017).
[48]
Robbert Van Renesse and Deniz Altinbuken. 2015. Paxos Made Moderately Complex. ACM Comput. Surv. Vol. 47, 3 (2015).
[49]
Robbert van Renesse, Nicolas Schiper, and Fred B. Schneider. 2015. Vive La Différence: Paxos vs. Viewstamped Replication vs. Zab. IEEE Trans. Dependable Sec. Comput. Vol. 12, 4 (2015).
[50]
Irene Zhang, Naveen Kr. Sharma, Adriana Szekeres, Arvind Krishnamurthy, and Dan R. K. Ports. 2015. Building Consistent Transactions with Inconsistent Replication SOSP.
[51]
Yang Zhang, Russell Power, Siyuan Zhou, Yair Sovran, Marcos K. Aguilera, and Jinyang Li. 2013. Transaction Chains: Achieving Serializability with Low Latency in Geo-distributed Storage Systems. In SOSP.

Cited By

View all
  • (2024)Hybrid Shared-Buffer for Multi-Master DatabasesJournal of Database Management10.4018/JDM.35692035:1(1-27)Online publication date: 7-Nov-2024
  • (2024)Distributed Transactions in Microservice Architecture: Informed Decision-making StrategiesVìsnik Nacìonalʹnogo unìversitetu "Lʹvìvsʹka polìtehnìka". Serìâ Ìnformacìjnì sistemi ta merežì10.23939/sisn2024.15.44915(449-459)Online publication date: 15-Jul-2024
  • (2024)Cloud Actor-Oriented Database Transactions in OrleansProceedings of the VLDB Endowment10.14778/3685800.368580117:12(3720-3730)Online publication date: 8-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGMOD '18: Proceedings of the 2018 International Conference on Management of Data
May 2018
1874 pages
ISBN:9781450347037
DOI:10.1145/3183713
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. distributed transactions
  2. globally-distributed data

Qualifiers

  • Research-article

Conference

SIGMOD/PODS '18
Sponsor:

Acceptance Rates

SIGMOD '18 Paper Acceptance Rate 90 of 461 submissions, 20%;
Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)35
  • Downloads (Last 6 weeks)4
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Hybrid Shared-Buffer for Multi-Master DatabasesJournal of Database Management10.4018/JDM.35692035:1(1-27)Online publication date: 7-Nov-2024
  • (2024)Distributed Transactions in Microservice Architecture: Informed Decision-making StrategiesVìsnik Nacìonalʹnogo unìversitetu "Lʹvìvsʹka polìtehnìka". Serìâ Ìnformacìjnì sistemi ta merežì10.23939/sisn2024.15.44915(449-459)Online publication date: 15-Jul-2024
  • (2024)Cloud Actor-Oriented Database Transactions in OrleansProceedings of the VLDB Endowment10.14778/3685800.368580117:12(3720-3730)Online publication date: 8-Nov-2024
  • (2024)Occam's Razor for Distributed ProtocolsProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698514(618-636)Online publication date: 20-Nov-2024
  • (2024)LMPT: A Novel Authenticated Data Structure to Eliminate Storage Bottlenecks for High Performance BlockchainsIEEE Transactions on Network and Service Management10.1109/TNSM.2023.334620221:2(1333-1343)Online publication date: Apr-2024
  • (2024)Managing Metaverse Data Tsunami: Actionable InsightsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.335496036:12(7423-7441)Online publication date: Dec-2024
  • (2024)GaussDB-Global: A Geographically Distributed Database System2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00383(5111-5118)Online publication date: 13-May-2024
  • (2024)An efficient sharding consensus protocol for improving blockchain scalabilityComputer Communications10.1016/j.comcom.2024.108032(108032)Online publication date: Dec-2024
  • (2023)Caerus: Low-Latency Distributed Transactions for Geo-Replicated SystemsProceedings of the VLDB Endowment10.14778/3632093.363210917:3(469-482)Online publication date: 1-Nov-2023
  • (2023)TiQuE: Improving the Transactional Performance of Analytical Systems for True Hybrid WorkloadsProceedings of the VLDB Endowment10.14778/3598581.359859816:9(2274-2288)Online publication date: 10-Jul-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media