extended-abstract

Public Access

Brief Announcement: What's Live? Understanding Distributed Consensus

Authors:

Yanhong A. LiuAuthors Info & Claims

PODC'21: Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing

Pages 565 - 568

https://doi.org/10.1145/3465084.3467947

Published: 23 July 2021 Publication History

Abstract

Distributed consensus algorithms such as Paxos have been studied extensively. Many different liveness properties and assumptions have been stated for them, but there are no systematic comparisons for better understanding of these properties.

This paper systematically studies and compares different liveness properties stated for over 30 prominent consensus algorithms and variants. We introduced a precise high-level language and formally specified these properties in the language. We then create a hierarchy of liveness properties combining two hierarchies of the assumptions used and a hierarchy of the assertions made, and compare the strengths and weaknesses of algorithms that ensure these properties. Our formal specifications and systematic comparisons led to the discovery of a range of problems in various stated liveness properties. We also developed TLA+ specifications of these liveness properties, and we used model checking of execution steps to illustrate liveness patterns for Paxos.

Supplementary Material

MP4 File (zoom_0.mp4)

Presentation video

Download
41.96 MB

References

[1]

Marcos Kawazoe Aguilera, Wei Chen, and Sam Toueg. 2000. Failure Detection and Consensus in the Crash-recovery Model. Distrib. Comput., Vol. 13, 2 (April 2000), 99--125. https://doi.org/10.1007/s004460050070

Digital Library

[2]

Yair Amir. 1995. Replication Using Group Communication Over a Partitioned Network. Ph.D. Dissertation. Hebrew University of Jerusalem, Jerusalem, Israel. Advisor(s) Dolev, Danny. http://www.dsn.jhu.edu/ yairamir/Yair_phd.pdf

[3]

Yair Amir, Louise E. Moser, Peter M. Melliar-Smith, eborah A. Agarwal, and Paul Ciarfella. 1995. The Totem Single-ring Ordering and Membership Protocol. ACM Trans. Comput. Syst., Vol. 13, 4 (Nov. 1995), 311--342. https://doi.org/10.1145/210223.210224

Digital Library

[4]

Yair Amir and Ciprian Tutu. 2001. From Total Order to Database Replication. Technical Report. Johns Hopkins University, Baltimore, MD, USA. http://www.dsn.jhu.edu/pub/papers/cnds-2001-6.pdf CNDS-2001-6.

[5]

Yair Amir and Ciprian Tutu. 2002. From Total Order to Database Replication. In Proceedings of the 22nd International Conference on Distributed Computing Systems (ICDCS '02). IEEE Computer Society, Los Alamitos, CA, USA, 494--503. https://doi.org/10.1109/ICDCS.2002.1022299

Digital Library

[6]

Kenneth P. Birman and Thomas A. Joseph. 1987 a. Exploiting Virtual Synchrony in Distributed Systems. SIGOPS Oper. Syst. Rev., Vol. 21, 5 (Nov. 1987), 123--138. https://doi.org/10.1145/37499.37515

Digital Library

[7]

Kenneth P. Birman and Thomas A. Joseph. 1987 b. Reliable Communication in the Presence of Failures. ACM Trans. Comput. Syst., Vol. 5, 1 (Jan. 1987), 47--76. https://doi.org/10.1145/7351.7478

Digital Library

[8]

Kenneth P. Birman, Dahlia Malkhi, and Robbert Van Renesse. 2010. Virtually Synchronous Methodology for Dynamic Service Replication. Technical Report MSR-TR-2010-151. Microsoft Research, Redmond, WA, USA. https://www.microsoft.com/en-us/research/wp-content/uploads/2010/11/vs-submit.pdf

[9]

Mike Burrows. 2006. The Chubby Lock Service for Loosely-coupled Distributed Systems. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (Seattle, Washington) (OSDI '06). USENIX Association, Berkeley, CA, USA, 335--350. http://dl.acm.org/citation.cfm?id=1298455.1298487

Digital Library

[10]

Saksham Chand and Yanhong A. Liu. 2020. What's live? Understanding Distributed Consensus. Computing Research Repository, Vol. normalfont arXiv:2001.04787 [cs.DC] (Jan. 2020). http://arxiv.org/abs/2001.04787

[11]

Saksham Chand, Yanhong A. Liu, and Scott D. Stoller. 2016. Formal Verification of Multi-Paxos for Distributed Consensus. In FM 2016: Formal Methods (Limassol, Cyprus) (FM '16). Springer International Publishing, Cham, Switzerland, 119--136. https://doi.org/10.1007/978-3-319-48989-6_8

[12]

Tushar D. Chandra, Robert Griesemer, and Joshua Redstone. 2007. Paxos Made Live: An Engineering Perspective. In Proceedings of the 26th Annual ACM Symposium on Principles of Distributed Computing (Portland, Oregon, USA) (PODC '07). ACM, New York, NY, USA, 398--407. https://doi.org/10.1145/1281100.1281103

Digital Library

[13]

Tushar Deepak Chandra and Sam Toueg. 1996. Unreliable Failure Detectors for Reliable Distributed Systems. J. ACM, Vol. 43, 2 (March 1996), 225--267. https://doi.org/10.1145/226643.226647

Digital Library

[14]

Roberto De Prisco, Butler Lampson, and Nancy Lynch. 2000. Revisiting the PAXOS Algorithm. Theor. Comput. Sci., Vol. 243, 1--2 (July 2000), 35--91. https://doi.org/10.1016/S0304-3975(00)00042-6

Digital Library

[15]

Cezara Druagoi, Thomas A. Henzinger, and Damien Zufferey. 2016. PSync: A Partially Synchronous Language for Fault-tolerant Distributed Algorithms. SIGPLAN Not., Vol. 51, 1 (Jan. 2016), 400--415. https://doi.org/10.1145/2914770.2837650

[16]

Álvaro Garc'ia-Pérez, Alexey Gotsman, Yuri Meshman, and Ilya Sergey. 2018a. Paxos Consensus, Deconstructed and Abstracted. In Programming Languages and Systems (Thessaloniki, Greece) (ESOP '18). Springer International Publishing, Cham, Switzerland, 912--939. https://doi.org/10.1007/978-3-319-89884-1_32

[17]

Álvaro Garc'ia-Pérez, Alexey Gotsman, Yuri Meshman, and Ilya Sergey. 2018b. Paxos Consensus, Deconstructed and Abstracted (Extended Version). Computing Research Repository, Vol. arXiv:1802.05969 (2018), 44 pages. https://arxiv.org/abs/1802.05969

[18]

Chris Hawblitzel, Jon Howell, Manos Kapritsos, Jacob R. Lorch, Bryan Parno, Michael L. Roberts, Srinath Setty, and Brian Zill. 2015. IronFleet: Proving Practical Distributed Systems Correct. In Proceedings of the 25th Symposium on Operating Systems Principles (Monterey, California) (SOSP '15). ACM, New York, NY, USA, 1--17. https://doi.org/10.1145/2815400.2815428

Digital Library

[19]

Sagar Jha, Jonathan Behrens, Theo Gkountouvas, Matthew Milano, Weijia Song, Edward Tremel, Robbert Van Renesse, Sydney Zink, and Kenneth P. Birman. 2019. Derecho: Fast State Machine Replication for Cloud Services. ACM Trans. Comput. Syst., Vol. 36, 2, Article 4 (April 2019), 49 pages. https://doi.org/10.1145/3302258

Digital Library

[20]

Flavio P. Junqueira, Benjamin C. Reed, and Marco Serafini. 2011. Zab: High-performance Broadcast for Primary-backup Systems. In Proceedings of the 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN '11). IEEE Computer Society, Washington, DC, USA, 245--256. https://doi.org/10.1109/DSN.2011.5958223

Digital Library

[21]

Idit Keidar. 1994. A Highly Available Paradigm for Consistent Object Replication. Master's thesis. The Hebrew University of Jerusalem. https://www.cs.huji.ac.il/labs/transis/Abstracts/keidar-msc.html CS95-5.

[22]

Idit Keidar and Danny Dolev. 1996. Efficient Message Ordering in Dynamic Networks. In Proceedings of the 15th Annual ACM Symposium on Principles of Distributed Computing (Philadelphia, Pennsylvania, USA) (PODC '96). ACM, New York, NY, USA, 68--76. https://doi.org/10.1145/248052.248062

Digital Library

[23]

Idit Keidar and Alexander Shraer. 2006 a. Timeliness, Failure-detectors, and Consensus Performance. In Proceedings of the 25th Annual ACM Symposium on Principles of Distributed Computing (Denver, Colorado, USA) (PODC '06). ACM, New York, NY, USA, 169--178. https://doi.org/10.1145/1146381.1146408

Digital Library

[24]

Idit Keidar and Alexander Shraer. 2006 b. Timeliness, Failure-detectors, and Consensus Performance. Technical Report. Technion - Israel Institute of Technology, Haifa, Israel. http://www.cs.technion.ac.il/ shralex/GIRAF-TR-Feb06.pdf CCIT 576.

[25]

Pertti Kellomäki. 2004. An annotated specification of the consensus protocol of Paxos using superposition in PVS. Report 36. Institute of Software Systems, Tampere University of Technology, Tampere, Finland. http://www.cs.tut.fi/ohj/laitosraportit/report36-paxos.pdf

[26]

Jonathan Kirsch and Yair Amir. 2008a. Paxos for system builders. Technical Report. Johns Hopkins University, Baltimore, MD, USA. http://www.cnds.jhu.edu/pub/papers/cnds-2008-2.pdf CNDS-2008-2.

[27]

Jonathan Kirsch and Yair Amir. 2008b. Paxos for System Builders: An Overview. In Proceedings of the 2nd Workshop on Large-Scale Distributed Systems and Middleware (Yorktown Heights, New York, USA) (LADIS '08). ACM, New York, NY, USA, Article 3, 6 pages. https://doi.org/10.1145/1529974.1529979

Digital Library

[28]

Leslie Lamport. 1998. The Part-time Parliament. ACM Trans. Comput. Syst., Vol. 16, 2 (May 1998), 133--169. https://doi.org/10.1145/279227.279229

Digital Library

[29]

Leslie Lamport. 2001. Paxos made simple. ACM SIGACT News, Vol. 32, 4 (Dec. 2001), 51--58. https://doi.org/10.1145/568425.568433

[30]

Leslie Lamport. 2006. Fast Paxos. Distributed Computing, Vol. 19, 2 (Oct. 2006), 79--103. https://doi.org/10.1007/s00446-006-0005-x

Digital Library

[31]

Leslie Lamport, Dahlia Malkhi, and Lidong Zhou. 2009. Vertical Paxos and Primary-backup Replication. In Proceedings of the 28th ACM Symposium on Principles of Distributed Computing (Calgary, AB, Canada) (PODC '09). ACM, New York, NY, USA, 312--313. https://doi.org/10.1145/1582716.1582783

Digital Library

[32]

Barbara Liskov and James Cowling. 2012. Viewstamped replication revisited. Technical Report. Massachusetts Institute of Technology, Cambridge, MA, USA. http://pmg.csail.mit.edu/papers/vr-revisited.pdf

[33]

Yanhong A. Liu, Saksham Chand, and Scott D. Stoller. 2019. Moderately Complex Paxos Made Simple: High-Level Executable Specification of Distributed Algorithms. In Proceedings of the 21st International Symposium on Principles and Practice of Programming Languages 2019 (Porto, Portugal) (PPDP '19). ACM, New York, NY, USA, Article 15, 15 pages. https://doi.org/10.1145/3354166.3354180

[34]

Yanhua Mao, Flavio P. Junqueira, and Keith Marzullo. 2008. Mencius: Building Efficient Replicated State Machines for WANs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (San Diego, California) (OSDI'08). USENIX Association, Berkeley, CA, USA, 369--384. http://dl.acm.org/citation.cfm?id=1855741.1855767

[35]

André Medeiros. 2012. ZooKeeper's atomic broadcast protocol: Theory and practice. Technical Report. Aalto University School of Science, Espoo, Finland. http://www.tcs.hut.fi/Studies/T-79.5001/reports/2012-deSouzaMedeiros.pdf

[36]

Iulian Moraru, David G. Andersen, and Michael Kaminsky. 2013. There is More Consensus in Egalitarian Parliaments. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (Farminton, Pennsylvania) (SOSP '13). ACM, New York, NY, USA, 358--372. https://doi.org/10.1145/2517349.2517350

Digital Library

[37]

Brian M. Oki and Barbara H. Liskov. 1988. Viewstamped Replication: A New Primary Copy Method to Support Highly-Available Distributed Systems. In Proceedings of the 7th Annual ACM Symposium on Principles of Distributed Computing (Toronto, Ontario, Canada) (PODC '88). ACM, New York, NY, USA, 8--17. https://doi.org/10.1145/62546.62549

[38]

Diego Ongaro and John Ousterhout. 2014. In Search of an Understandable Consensus Algorithm. In Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference (Philadelphia, PA) (USENIX ATC'14). USENIX Association, Berkeley, CA, USA, 305--320. https://www.usenix.org/conference/atc14/technical-sessions/presentation/ongaro

[39]

Oded Padon, Giuliano Losa, Mooly Sagiv, and Sharon Shoham. 2017. Paxos Made EPR: Decidable Reasoning About Distributed Protocols. Proc. ACM Program. Lang., Vol. 1, OOPSLA, Article 108 (Oct. 2017), 31 pages. https://doi.org/10.1145/3140568

Digital Library

[40]

Robbert Van Renesse and Deniz Altinbuken. 2015. Paxos Made Moderately Complex. ACM Comput. Surv., Vol. 47, 3, Article 42 (Feb. 2015), 36 pages. https://doi.org/10.1145/2673577

Digital Library

[41]

James R. Wilcox, Ilya Sergey, and Zachary Tatlock. 2017. Programming Language Abstractions for Modularly Verified Distributed Systems. In 2nd Summit on Advances in Programming Languages (SNAPL 2017). Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 19:1--19:12. https://doi.org/10.4230/LIPIcs.SNAPL.2017.19

Cited By

Liu YStoller S(2024)Tutorial: Consensus Algorithms from Classical to Blockchain: Quickly Program, Configure, Run, and Check2024 IEEE 44th International Conference on Distributed Computing Systems Workshops (ICDCSW)10.1109/ICDCSW63686.2024.00005(1-4)Online publication date: 23-Jul-2024
https://doi.org/10.1109/ICDCSW63686.2024.00005
Shivam KPaladugu VLiu YLiu YSchiller E(2023)Specification and Runtime Checking of Derecho, A Protocol for Fast Replication for Cloud ServicesProceedings of the 5th workshop on Advanced tools, programming languages, and PLatforms for Implementing and Evaluating algorithms for Distributed systems10.1145/3584684.3597275(1-10)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3584684.3597275
Paul SAgha GPatterson SVarela C(2022)Eventual consensus in Synod: verification using a failure-aware actor modelInnovations in Systems and Software Engineering10.1007/s11334-022-00463-519:4(395-410)Online publication date: 25-Jul-2022
https://dl.acm.org/doi/10.1007/s11334-022-00463-5

Index Terms

Brief Announcement: What's Live? Understanding Distributed Consensus
1. Computing methodologies
  1. Distributed computing methodologies

Recommendations

Formal Verification of Consensus in the Taurus Distributed Database
Formal Methods
Abstract
Distributed database services are an increasingly important part of cloud computing. They are required to satisfy several key properties, including consensus and fault tolerance. Given the highly concurrent nature of these systems, subtle errors ... $^{}$
On the refinement of liveness properties of distributed systems

We present a new approach, based on simulation relations, for reasoning about liveness properties of distributed systems. Our contribution consists of (1) a formalism for defining liveness properties, (2) a proof method for liveness properties based on ...
Probably safe or live
CSL-LICS '14: Proceedings of the Joint Meeting of the Twenty-Third EACSL Annual Conference on Computer Science Logic (CSL) and the Twenty-Ninth Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)

This paper presents a formal characterisation of safety and liveness properties for fully probabilistic systems. As for the classical setting, it is established that any (probabilistic tree) property is equivalent to a conjunction of a safety and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PODC'21: Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing

July 2021

590 pages

ISBN:9781450385480

DOI:10.1145/3465084

General Chair:
Avery Miller
University of Manitoba, Canada
,
Program Chair:
Keren Censor-Hillel
Technion, Israel

Copyright © 2021 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2021

Check for updates

Author Tags

Qualifiers

Extended-abstract

Funding Sources

Conference

PODC '21

Sponsor:

PODC '21: ACM Symposium on Principles of Distributed Computing

July 26 - 30, 2021

Virtual Event, Italy

Acceptance Rates

Overall Acceptance Rate 740 of 2,477 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
380
Total Downloads

Downloads (Last 12 months)143
Downloads (Last 6 weeks)14

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu YStoller S(2024)Tutorial: Consensus Algorithms from Classical to Blockchain: Quickly Program, Configure, Run, and Check2024 IEEE 44th International Conference on Distributed Computing Systems Workshops (ICDCSW)10.1109/ICDCSW63686.2024.00005(1-4)Online publication date: 23-Jul-2024
https://doi.org/10.1109/ICDCSW63686.2024.00005
Shivam KPaladugu VLiu YLiu YSchiller E(2023)Specification and Runtime Checking of Derecho, A Protocol for Fast Replication for Cloud ServicesProceedings of the 5th workshop on Advanced tools, programming languages, and PLatforms for Implementing and Evaluating algorithms for Distributed systems10.1145/3584684.3597275(1-10)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3584684.3597275
Paul SAgha GPatterson SVarela C(2022)Eventual consensus in Synod: verification using a failure-aware actor modelInnovations in Systems and Software Engineering10.1007/s11334-022-00463-519:4(395-410)Online publication date: 25-Jul-2022
https://dl.acm.org/doi/10.1007/s11334-022-00463-5
Iosif EChristodoulou KTouloupou MInglezakis A(2020)Leadership Uniformity in Raft Consensus AlgorithmInformation Systems10.1007/978-3-030-63396-7_9(125-136)Online publication date: 21-Nov-2020
https://doi.org/10.1007/978-3-030-63396-7_9

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten