Laura Lawniczak
ABSTRACT
Stream-based replication facilitates the deployment and operation of state-machine replication protocols by running them as applications on top of data-stream processing frameworks. Taking advantage of platform-provided features, this approach makes it possible to significantly minimize implementation complexity at the protocol level. To further extend the associated benefits, in this paper we examine how the concept can be used to provide generic support for creating, storing, and applying checkpoints of replica states, both in the use case for catch up and garbage collection as well as to recover failed replicas. Specifically, we present three checkpointing-mechanism designs with different degrees of platform involvement and evaluate them in the context of Twitter's stream-processing engine Heron.
- Eduardo Alchieri, Fernando Dotti, Odorico M Mendizabal, and Fernando Pedone. 2017. Reconfiguring Parallel State Machine Replication. In Proceedings of the 36th International Symposium on Reliable Distributed Systems (SRDS '17). 104--113.Google Scholar
Cross Ref
- Alysson Bessani, Marcel Santos, João Felix, Nuno Neves, and Miguel Correia. 2013. On the Efficiency of Durable State Machine Replication. In Proceedings of the 2013 USENIX Annual Technical Conference (USENIX ATC '13). 169--180.Google Scholar
- Alysson Bessani, João Sousa, and Eduardo E P Alchieri. 2014. State Machine Replication for the Masses with BFT-SMaRt. In Proceedings of the 44th International Conference on Dependable Systems and Networks (DSN '14). 355--362.Google ScholarDigital Library
- Tobias Distler. 2021. Byzantine Fault-Tolerant State-Machine Replication from a Systems Perspective. Comput. Surveys 54, 1, Article 24 (2021), 38 pages.Google Scholar
- Tobias Distler, Rüdiger Kapitza, Ivan Popov, Hans P. Reiser, and Wolfgang Schröder-Preikschat. 2011. SPARE: Replicas on Hold. In Proceedings of the 18th Network and Distributed System Security Symposium (NDSS '11). 407--420.Google Scholar
- Tobias Distler, Rüdiger Kapitza, and Hans P. Reiser. 2010. State Transfer for Hypervisor-Based Proactive Recovery of Heterogeneous Replicated Services. In Proceedings of the 5th "Sicherheit, Schutz und Zuverlässigkeit" Conference (SICHERHEIT '10). 61--72.Google Scholar
- Michael Eischer, Markus Büttner, and Tobias Distler. 2019. Deterministic Fuzzy Checkpoints. In Proceedings of the 38th International Symposium on Reliable Distributed Systems (SRDS '19). 153--162.Google ScholarCross Ref
- E. N. (Mootaz) Elnozahy, Lorenzo Alvisi, Yi-Min Wang, and David B. Johnson. 2002. A Survey of Rollback-Recovery Protocols in Message-Passing Systems. Comput. Surveys 34, 3 (2002), 375--408.Google ScholarDigital Library
- Robert Hagmann. 1987. Reimplementing the Cedar File System Using Logging and Group Commit. In Proceedings of the 11th Symposium on Operating Systems Principles (SOSP '87). 155--162.Google ScholarDigital Library
- Jaehyun Hwang and Qizhe Cai. 2020. TCP ≈ RDMA: CPU-efficient Remote Storage Access with i10. In Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI '20). 127--140.Google Scholar
- Jan Kończak and Paweł T Wojciechowski. 2021. Failure Recovery from Persistent Memory in Paxos-Based State Machine Replication. In Proceedings of the 40th International Symposium on Reliable Distributed Systems (SRDS '21). 88--98.Google ScholarCross Ref
- Jan Kończak, Paweł T Wojciechowski, Nuno Santos, Tomasz Żurkowski, and André Schiper. 2019. Recovery Algorithms for Paxos-Based State Machine Replication. IEEE Transactions on Dependable and Secure Computing 18, 2 (2019), 623--640.Google ScholarDigital Library
- Sanjeev Kulkarni, Nikunj Bhagat, Maosong Fu, Vikas Kedigehalli, Christopher Kellogg, Sailesh Mittal, Jignesh M. Patel, Karthik Ramasamy, and Siddarth Taneja. 2015. Twitter Heron: Stream Processing at Scale. In Proceedings of the 41st International Conference on Management of Data (SIGMOD '15). 239--250.Google ScholarDigital Library
- Leslie Lamport. 1998. The Part-time Parliament. ACM Transactions on Computer Systems 16, 2 (1998), 133--169.Google ScholarDigital Library
- Laura Lawniczak and Tobias Distler. 2021. Stream-based State Machine Replication. In Proceedings of the 17th European Dependable Computing Conference (EDCC '21). 119--126.Google ScholarCross Ref
- Xiaojian Liao, Zhe Yang, and Jiwu Shu. 2022. RIO: Order-Preserving and CPU-Efficient Remote Storage Access. arXiv preprint arXiv:2210.08934 (2022).Google Scholar
- Odorico M Mendizabal, Fernando Luís Dotti, and Fernando Pedone. 2016. Analysis of Checkpointing Overhead in Parallel State Machine Replication. In Proceedings of the 31st Annual ACM Symposium on Applied Computing (SAC '16). 534--537.Google ScholarDigital Library
- Odorico M Mendizabal, Parisa Jalili Marandi, Fernando Luís Dotti, and Fernando Pedone. 2014. Checkpointing in Parallel State-Machine Replication. In Proceedings of the 18th International Conference on Principles of Distributed Systems (OPODIS '14). 123--138.Google ScholarCross Ref
- Diego Ongaro and John Ousterhout. 2014. In Search of an Understandable Consensus Algorithm. In Proceedings of the 2014 USENIX Annual Technical Conference (USENIX ATC '14). 305--320.Google Scholar
- Tuanir F Rezende, Pierre Sutra, Rodrigo Q Saramago, and Lasaro Camargos. 2017. On Making Generalized Paxos Practical. In Proceedings of the 31st International Conference on Advanced Information Networking and Applications (AINA '17). 347--354.Google ScholarCross Ref
- Ankit Toshniwal, Siddarth Taneja, Amit Shukla, Karthik Ramasamy, Jignesh M Patel, Sanjeev Kulkarni, Jason Jackson, Krishna Gade, Maosong Fu, Jake Donham, Nikunj Bhagat, Sailesh Mittal, and Dmitriy Ryaboy. 2014. Storm @Twitter. In Proceedings of the 40th International Conference on Management of Data (SIGMOD '14). 147--156.Google ScholarDigital Library
- Qingfeng Zhuge, Hao Zhang, Edwin Hsing-Mean Sha, Rui Xu, Jun Liu, and Shengyu Zhang. 2021. Exploring Efficient Architectures on Remote In-Memory NVM over RDMA. ACM Transactions on Embedded Computing Systems (TECS) 20, 5s (2021), 1--20.Google ScholarDigital Library
Index Terms
- Generic Checkpointing Support for Stream-based State-Machine Replication
Recommendations
Towards Energy-Proportional State-Machine Replication
ARM 2015: Proceedings of the 14th International Workshop on Adaptive and Reflective MiddlewareThe energy consumption of state-of-the-art systems applying state-machine replication in general is not proportional to the performance they provide. This is mainly due to the fact that current implementations rely on static replica configurations, for ...
An enhanced model-based checkpointing protocol
PDCN'07: Proceedings of the 25th conference on Proceedings of the 25th IASTED International Multi-Conference: parallel and distributed computing and networksCheckpointing and rollback recovery are widely used techniques to handle failures in distributed computing systems. Usually we avoid taking checkpoints that are useless during the recovery process. Communication-Induced checkpointing algorithms ...
Scalable Byzantine fault-tolerant state-machine replication on heterogeneous servers
When provided with more powerful or extra hardware, state-of-the-art Byzantine fault-tolerant (BFT) replication protocols are unable to effectively exploit the additional computing resources: on the one hand, in settings with heterogeneous servers ...
Comments