Abstract
Log replication is the key component of high available database system. To guarantee data consistency and reliability, modern database systems often use Paxos protocol to replicate log in multiple database instance sites. Since the replicated logs need to contain some metadata such as committed log sequence number (LSN), this increases the overhead of storage and network. It has significantly negative impact on the throughput in the update intensive work load. In this paper, we present an implementation of log replication and database recovery, which adopts the idea of piggybacking, i.e. committed LSN is embedded in the commit logs. This practice not only retains virtues of Paxos replication, but also reduces disk and network IO effectively, which enhances performance and decreases recovery time. We implemented and evaluated our approach in a main memory database system (Oceanbase), and found that our method can offer 1.3x higher throughput than traditional log replication with synchronization mechanism.
This work is partially supported by National High-tech R&D Program (863 Program) under grant number 2015AA015307, and National Science Foundation of China under grant number 61332006.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Cattell, R.: Scalable SQL and noSQL data stores. SIGMOD Rec. 39(4), 12–27 (2010)
Stonebraker, M., Cetintemel, U.: “One size fits all”: an idea whose time has come and gone. In: Proceedings of ICDE, pp. 2–11 (2005)
Lamport, L.: The part-time parliament. TOCS 16(2), 133–169 (1998)
OceanBase website. https://github.com/alibaba/oceanbase
Mohan, C., et al.: ARIES: a transaction recovery method supporting fine-granularity locking and partial roll backs using write-ahead logging. TODS 17(1), 94–162 (1992)
DeCandia G., Hastorun D., Jampani M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: amazons highlyavailable key-value store. In: Proceedings of SOSP, pp. 205–220 (2007)
Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. SIGOPS 44(2), 35–40 (2010)
Cassandra website. http://cassandra.apache.org/
Cooper, B.F., Ramakrishnan, R., Srivastava, U., et al.: PUNTS: Yahoo!’s hosted data serving platform. In: Proceedings of VLDB, pp. 1277–1288 (2008)
Lamport, L.: Paxos made simple. SIGACT 32(4), 18–25 (2001)
Lamport, L.: Fast paxos. Distrib. Comput. 19(2), 79–103 (2006)
Ongard, D., Ousterhout, J.: In search of an understandable consensus algorithm. In: Proceedings of ATC (2014)
Raft consensus algorithm website. https://raft.github.io
Ousterhout, J., Agrawal, P., Erikson, D., et al.: The case for RAMCloud. CACM 54, 121–130 (2011)
Burrows, M.: The chubby lock service for loosely coupled distributed systems. In: Proceedings of OSDI, pp. 335–350 (2006)
Zookeeper website. https://zookeeper.apache.org/
Baker, J., Bond, C., Corbett, J.C., Megastore, et al.: Providing scalable, highly available storage for interactive services. In: Proceedings of CIDR, pp. 223–234 (2011)
Corbett J.C., Dean J., Epstein, M.: Spanner: Googles globally distributed database. In: Proceedings of OSDI (2012)
Shute, J., Vingralek, R., Bart, S., et al.: F1: a distributed SQL database that scales. In: Proceedings of VLDB, pp. 1068–1079 (2013)
Rao, J., Shekita, E.J., Tata, S.: Using paxos to bulid a scalable, consistent, highly available datastore. In: Proceedings of VLDB, pp. 243–254 (2011)
Patterson, S., et al.: Serializability, not serial: concurrency control and availability in multi-datacenter datastores. Proc. VLDB Endow. 5(11), 1459–1470 (2012)
Dragojevic, A., Narayanan, D., et al.: No compromises: distributed transactions with consistency, availability, and performance. In: Proceedings of SOSP, pp. 54–70 (2015)
Thomson, A., Diamond, T., et al.: Calvin: fast distributed transactions for partitioned database systems. In: Proceedings of SIGMOD, pp. 1–12 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Guo, J., Zhang, C., Cai, P., Zhou, M., Zhou, A. (2016). Low Overhead Log Replication for Main Memory Database System. In: Cui, B., Zhang, N., Xu, J., Lian, X., Liu, D. (eds) Web-Age Information Management. WAIM 2016. Lecture Notes in Computer Science(), vol 9659. Springer, Cham. https://doi.org/10.1007/978-3-319-39958-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-39958-4_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-39957-7
Online ISBN: 978-3-319-39958-4
eBook Packages: Computer ScienceComputer Science (R0)