Hostname: page-component-8448b6f56d-cfpbc Total loading time: 0 Render date: 2024-04-23T13:43:29.455Z Has data issue: false hasContentIssue false
  • English
  • Français

Protocol combinators for modeling, testing, and execution of distributed systems

Part of: PADL 2019 Conference

Published online by Cambridge University Press:  15 February 2021

KRISTOFFER JUST ARNDAL ANDERSEN Open the ORCID record for KRISTOFFER JUST ARNDAL ANDERSEN [Opens in a new window]  and
ILYA SERGEY
KRISTOFFER JUST ARNDAL ANDERSEN
Affiliation:
Department of Computer Science, Aarhus University, Aarhus, Denmark (e-mail: kristoffer@arndalandersen.dk)
ILYA SERGEY
Affiliation:
NUS School of Computing, Yale-NUS College and National University of Singapore, Singapore (e-mail: ilya.sergey@yale-nus.edu.sg)
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Distributed systems are hard to get right, model, test, debug, and teach. Their textbook definitions, typically given in a form of replicated state machines, are concise, yet prone to introducing programming errors if naïvely translated into runnable implementations.

In this work, we present Distributed Protocol Combinators (DPC), a declarative programming framework that aims to bridge the gap between specifications and runnable implementations of distributed systems, and facilitate their modeling, testing, and execution. DPC builds on the ideas from the state-of-the art logics for compositional systems verification. The contribution of DPC is a novel family of program-level primitives, which facilitates construction of larger distributed systems from smaller components, streamlining the usage of the most common asynchronous message-passing communication patterns, and providing machinery for testing and user-friendly dynamic verification of systems. This paper describes the main ideas behind the design of the framework and presents its implementation in Haskell. We introduce DPC through a series of characteristic examples and showcase it on a number of distributed protocols from the literature.

This paper extends our preceeding conference publication (Andersen & Sergey, 2019a) with an exploration of randomized testing for protocols and their implementations, and an additional case study demonstrating bounded model checking of protocols.

Type
Research Article
Information
Journal of Functional Programming , Volume 31 , 2021 , e3
Check for updates
Copyright
© The Author(s), 2021. Published by Cambridge University Press

References

Adamsen, , Christoffer, Quist, Møller, Anders, Karim, Rezwana, Sridharam, Manu, Tip, Frank & Sen, Koushik. (2017). Repairing event race errors by controlling nondeterminism. Pages 289–299 of: ICSE. ACM.Google Scholar
Ancona, Davide, Bono, Viviana, Bravetti, Mario, Campos, Joana, Castagna, Giuseppe, Deniélou, Pierre-Malo, Gay, Simon J., Gesbert, Nils, Giachino, Elena, Hu, Raymond, Johnsen, Einar Broch, Martins, Francisco, Mascardi, Viviana, Montesi, Fabrizio, Neykova, Rumyana, Ng, Nicholas, Padovani, Luca, Vasconcelos, Vasco T. & Yoshida, Nobuko. (2016). Behavioral types in programming languages. Foundations and trends in programming languages, 3(2-3), 95–230.CrossRefGoogle Scholar
Andersen, Kristoffer Just Arndal & Sergey, Ilya. (2019a). Distributed protocol combinators. Pages 169–186 of: PADL. LNCS, vol. 11372. Springer.CrossRefGoogle Scholar
Andersen, Kristoffer Just Arndal & Sergey, Ilya. (2019b). Distributed protocol combinators: Implementation. https://doi.org/10.5281/zenodo.3902686.CrossRefGoogle Scholar
Brady, Edwin. (2017). Type-driven development of concurrent communicating systems. Computer science (AGH), 18(3).Google Scholar
Chandra, Tushar, Griesemer, Robert & Redstone, Joshua. (2007). Paxos made live: an engineering perspective. Pages 398–407 of: PODC. ACM.Google Scholar
Claessen, Koen & Hughes, John. (2011). Quickcheck: a lightweight tool for random testing of haskell programs. Acm sigplan notices, 46(4), 53–64.CrossRefGoogle Scholar
Coq Development Team. (2020). The Coq Proof Assistant Reference Manual. Available from http://coq.inria.fr.Google Scholar
Desai, Ankush, Gupta, Vivek, Jackson, Ethan K., Qadeer, Shaz, Rajamani, Sriram K. & Zufferey, Damien. (2013). P: safe asynchronous event-driven programming. Pages 321–332 of: PLDI. ACM.Google Scholar
Desai, Ankush, Phanishayee, Amar, Qadeer, Shaz & Seshia, Sanjit. (2018). Compositional Programming and Testing of Dynamic Distributed Systems. PACMPL, 2(OOPSLA), 159:1–159:30.Google Scholar
Dinsdale-Young, Thomas, Dodds, Mike, Gardner, Philippa, Parkinson, Matthew J. & Vafeiadis, Viktor. (2010). Concurrent Abstract Predicates. Pages 504–528 of: ECOOP. LNCS, vol. 6183. Springer.Google Scholar
Dragoi, Cezara, Henzinger, Thomas A. & Zufferey, Damien. (2016). PSync: a partially synchronous language for fault-tolerant distributed algorithms. Pages 400–415 of: POPL. ACM.Google Scholar
Findler, Robert Bruce & Felleisen, Matthias. (2002). Contracts for higher-order functions. Pages 48–59 of: ICFP. ACM.Google Scholar
Garca-Pérez, Álvaro, Gotsman, Alexey, Meshman, Yuri & Sergey, Ilya. (2018). Paxos Consensus, Deconstructed and Abstracted. Pages 912–939 of: ESOP. LNCS, vol. 10801. Springer.Google Scholar
Gommerstadt, Hannah, Jia, Limin & Pfenning, Frank. (2018). Session-typed concurrent contracts. Pages 771–798 of: ESOP. LNCS, vol. 10801. Springer.Google Scholar
Gray, James N. (1978). Notes on data base operating systems. Pages 393–481 of: In Operating Systems. Springer.Google Scholar
Hawblitzel, Chris, Howell, Jon, Kapritsos, Manos, Lorch, Jacob R., Parno, Bryan, Roberts, Michael L., Setty, Srinath T. V. & Zill, Brian. (2015). IronFleet: proving practical distributed systems correct. Pages 1–17 of: SOSP. ACM.Google Scholar
Hüttel, Hans, Lanese, Ivan, Vasconcelos, Vasco T., Caires, Lus, Carbone, Marco, Deniélou, Pierre-Malo, Mostrous, Dimitris, Padovani, Luca, Ravara, António, Tuosto, Emilio, Vieira, Hugo Torres & Zavattaro, Gianluigi. (2016). Foundations of session types and behavioural contracts. ACM comput. surv., 49(1), 3:1–3:36.CrossRefGoogle Scholar
Jones, Cliff B. (1983). Tentative steps toward a development method for interfering programs. Acm transactions on programming languages and systems, 5(4), 596–619.CrossRefGoogle Scholar
Killian, Charles Edwin, Anderson, James W., Braud, Ryan, Jhala, Ranjit & Vahdat, Amin M. (2007). Mace: Language support for building distributed systems. Pages 179–188 of: PLDI. ACM.Google Scholar
Kleppmann, Martin. 2016 (Feb). How to do distributed locking. https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html.Google Scholar
Krogh-Jespersen, Morten, Timany, Amin, Ohlenbusch, Marit Edna, Gregersen, Simon Oddershede & Birkedal, Lars. (2020). Aneris: A mechanised logic for modular reasoning about distributed systems. Pages 336–365 of: ESOP. LNCS, vol. 12075. Springer.Google Scholar
Lamport, Leslie. (1998). The Part-Time Parliament. ACM toplas, 16(2), 133–169.Google Scholar
Lamport, Leslie. (2001). Paxos made simple.Google Scholar
Lamport, Leslie & Schneider, Fred B. (1985). Formal foundation for specification and verification. Pages 203–285 of: Distributed Systems: Methods and Tools for Specification, An Advanced Course. LNCS, vol. 190. Springer.Google Scholar
Lampson, Butler W. (1996). How to build a highly available system using consensus. WDAG.CrossRefGoogle Scholar
Lange, Julien & Tuosto, Emilio. (2012). Synthesising Choreographies from Local Session Types. Pages 225–239 of: CONCUR. LNCS, vol. 7454. Springer.Google Scholar
Leonini, Lorenzo, Riviere, Etienne & Felber, Pascal. (2009). SPLAY: distributed systems evaluation made simple (or how to turn ideas into live systems in a breeze). Pages 185–198 of: NSDI. USENIX Association.Google Scholar
Liang, Sheng, Hudak, Paul & Jones, Mark P. (1995). Monad transformers and modular interpreters. Pages 333–343 of: POPL. ACM Press.Google Scholar
Liu, Yanhong A., Stoller, Scott D., Lin, Bo & Gorbovitski, Michael. (2012). From clarity to efficiency for distributed algorithms. Pages 395–410 of: OOPSLA. ACM.Google Scholar
Melgratti, Hernán C. & Padovani, Luca. (2017). Chaperone contracts for higher-order sessions. Proc. ACM program. lang., 1(ICFP), 35:1–35:29.Google Scholar
Nanevski, Aleksandar, Morrisett, Greg, Shinnar, Avi, Govereau, Paul & Birkedal, Lars. (2008). Ynot: Dependent types for imperative programs. Pages 229–240 of: ICFP.Google Scholar
Newcombe, Chris, Rath, Tim, Zhang, Fan, Munteanu, Bogdan, Brooker, Marc & Deardeuff, Michael. (2015). How Amazon web services uses formal methods. Commun. ACM, 58(4).CrossRefGoogle Scholar
O’Hearn, Peter W., Reynolds, John C. & Yang, Hongseok. (2001). Local reasoning about programs that alter data structures. CSL. LNCS, vol. 2142. Springer.Google Scholar
Padon, Oded, McMillan, Kenneth L., Panda, Aurojit, Sagiv, Mooly & Shoham, Sharon. (2016). Ivy: safety verification by interactive generalization. Pages 614–630 of: PLDI. ACM.Google Scholar
Pfenning, Frank & Elliott, Conal. (1988). Higher-order abstract syntax. Pages 199–208 of: PLDI. ACM.Google Scholar
Prlea, George & Sergey, Ilya. (2018). Mechanising blockchain consensus. Pages 78–90 of: CPP. ACM.Google Scholar
Sergey, Ilya, Nanevski, Aleksandar, Banerjee, Anindya & Delbianco, Germán Andrés. (2016). Hoare-style Specifications as Correctness Conditions for Non-linearizable Concurrent Objects. Pages 92–110 of: OOPSLA. ACM.Google Scholar
Sergey, Ilya, Wilcox, James R. & Tatlock, Zachary. (2018). Programming and proving with distributed protocols. PACMPL, 2(POPL), 28:1–28:30.Google Scholar
van Renesse, Robbert & Altinbuken, Deniz. (2015). Paxos made moderately complex. ACM comp. surv., 47(3), 42:1–42:36.CrossRefGoogle Scholar
Weikum, Gerhard & Vossen, Gottfried. (2002). Transactional information systems: Theory, algorithms, and the practice of concurrency control and recovery. Morgan Kaufmann.Google Scholar
Wilcox, James R., Woos, Doug, Panchekha, Pavel, Tatlock, Zachary, Wang, Xi, Ernst, Michael D. & Anderson, Thomas E. (2015). Verdi: a framework for implementing and formally verifying distributed systems. Pages 357–368 of: PLDI. ACM.Google Scholar
Wilcox, James R., Sergey, Ilya & Tatlock, Zachary. (2017). Programming Language Abstractions for Modularly Verified Distributed Systems. Pages 19:1–19:12 of: SNAPL.Google Scholar
Submit a response

Discussions

No Discussions have been published for this article.