skip to main content
10.1145/3492321.3519594acmconferencesArticle/Chapter ViewAbstractPublication PageseurosysConference Proceedingsconference-collections

Narwhal and Tusk: a DAG-based mempool and efficient BFT consensus

Published:28 March 2022Publication History

ABSTRACT

We propose separating the task of reliable transaction dissemination from transaction ordering, to enable high-performance Byzantine fault-tolerant quorum-based consensus. We design and evaluate a mempool protocol, Narwhal, specializing in high-throughput reliable dissemination and storage of causal histories of transactions. Narwhal tolerates an asynchronous network and maintains high performance despite failures. Narwhal is designed to easily scale-out using multiple workers at each validator, and we demonstrate that there is no foreseeable limit to the throughput we can achieve.

Composing Narwhal with a partially synchronous consensus protocol (Narwhal-HotStuff) yields significantly better throughput even in the presence of faults or intermittent loss of liveness due to asynchrony. However, loss of liveness can result in higher latency. To achieve overall good performance when faults occur we design Tusk, a zero-message overhead asynchronous consensus protocol, to work with Narwhal. We demonstrate its high performance under a variety of configurations and faults.

As a summary of results, on a WAN, Narwhal-Hotstuff achieves over 130,000 tx/sec at less than 2-sec latency compared with 1,800 tx/sec at 1-sec latency for Hotstuff. Additional workers increase throughput linearly to 600,000 tx/sec without any latency increase. Tusk achieves 160,000 tx/sec with about 3 seconds latency. Under faults, both protocols maintain high throughput, but Narwhal-HotStuff suffers from increased latency.

References

  1. Daniel J Abadi and Jose M Faleiro. 2018. An overview of deterministic database systems. Commun. ACM 61, 9 (2018), 78--88.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Michael Abd-El-Malek, Gregory R. Ganger, Garth R. Goodson, Michael K. Reiter, and Jay J. Wylie. 2005. Fault-scalable Byzantine fault-tolerant services. In Proceedings of the 20th ACM Symposium on Operating Systems Principles 2005, SOSP 2005, Brighton, UK, October 23--26, 2005, Andrew Herbert and Kenneth P. Birman (Eds.). ACM, 59--74.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Ittai Abraham, Dahlia Malkhi, and Alexander Spiegelman. 2019. Asymptotically Optimal Validated Asynchronous Byzantine Agreement. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, PODC 2019, Toronto, ON, Canada, July 29 - August 2, 2019, Peter Robinson and Faith Ellen (Eds.). ACM, 337--346.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mustafa Al-Bassam, Alberto Sonnino, Shehar Bano, Dave Hrycyszyn, and George Danezis. 2018. Chainspace: A Sharded Smart Contracts Platform. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18--21, 2018. The Internet Society.Google ScholarGoogle Scholar
  5. Salem Alqahtani and Murat Demirbas. 2021. Bottlenecks in Blockchain Consensus Protocols. CoRR abs/2103.04234 (2021).Google ScholarGoogle Scholar
  6. Yair Amir, Brian Coan, Jonathan Kirsch, and John Lane. 2010. Prime: Byzantine replication under attack. IEEE transactions on dependable and secure computing 8, 4 (2010), 564--577.Google ScholarGoogle Scholar
  7. Elli Androulaki, Artem Barger, Vita Bortnikov, Christian Cachin, Konstantinos Christidis, Angelo De Caro, David Enyeart, Christopher Ferris, Gennady Laventman, Yacov Manevich, Srinivasan Muralidharan, Chet Murthy, Binh Nguyen, Manish Sethi, Gari Singh, Keith Smith, Alessandro Sorniotti, Chrysoula Stathakopoulou, Marko Vukolic, Sharon Weed Cocco, and Jason Yellick. 2018. Hyperledger fabric: a distributed operating system for permissioned blockchains. In Proceedings of the Thirteenth EuroSys Conference, EuroSys 2018, Porto, Portugal, April 23--26, 2018, Rui Oliveira, Pascal Felber, and Y. Charlie Hu (Eds.). ACM, 30:1--30:15.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Georgia Avarikioti, Eleftherios Kokoris-Kogias, and Roger Wattenhofer. 2021. Divide and Scale: Formalization of Distributed Ledger Sharding Protocols. arXiv:1910.10434 [cs.DC]Google ScholarGoogle Scholar
  9. Vivek Bagaria, Sreeram Kannan, David Tse, Giulia Fanti, and Pramod Viswanath. 2019. Prism: Deconstructing the blockchain to approach physical limits. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 585--602.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Leemon Baird. 2016. The swirlds hashgraph consensus algorithm: Fair, fast, byzantine fault tolerance. Swirlds Tech Reports SWIRLDS-TR-2016-01, Tech. Rep (2016).Google ScholarGoogle Scholar
  11. Shehar Bano, Alberto Sonnino, Andrey Chursin, Dmitri Perelman, and Dahlia Malkhi. 2020. Twins: White-Glove Approach for BFT Testing. arXiv preprint arXiv:2004.10617 (2020).Google ScholarGoogle Scholar
  12. Mathieu Baudet, Avery Ching, Andrey Chursin, George Danezis, François Garillot, Zekun Li, Dahlia Malkhi, Oded Naor, Dmitri Perelman, and Alberto Sonnino. 2019. State machine replication in the Libra blockchain. The Libra Assn., Tech. Rep (2019).Google ScholarGoogle Scholar
  13. Alysson Bessani, Joao Sousa, and Eduardo EP Alchieri. 2014. State machine replication for the masses with BFT-SMART. In 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. IEEE, 355--362.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Dan Boneh, Ben Lynn, and Hovav Shacham. 2004. Short Signatures from the Weil Pairing. J. Cryptol. 17, 4 (2004), 297--319.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Gabriel Bracha. 1987. Asynchronous Byzantine agreement protocols. Information and Computation 75, 2 (1987), 130--143.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Gabriel Bracha and Sam Toueg. 1985. Asynchronous consensus and broadcast protocols. Journal of the ACM (JACM) 32, 4 (1985), 824--840.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Ethan Buchman. 2016. Tendermint: Byzantine fault tolerance in the age of blockchains.Google ScholarGoogle Scholar
  18. Christian Cachin, Rachid Guerraoui, and Luís Rodrigues. 2011. Introduction to reliable and secure distributed programming. Springer Science & Business Media.Google ScholarGoogle Scholar
  19. Miguel Castro and Barbara Liskov. 2002. Practical byzantine fault tolerance and proactive recovery. ACM Trans. Comput. Syst. 20, 4 (2002), 398--461. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. George Danezis and David Hrycyszyn. 2018. Blockmania: from block DAGs to consensus. arXiv preprint arXiv:1809.01620 (2018).Google ScholarGoogle Scholar
  21. Danny Dolev, Michael J Fischer, Rob Fowler, Nancy A Lynch, and H Raymond Strong. 1982. An efficient algorithm for Byzantine agreement without authentication. Information and Control 52, 3 (1982), 257--274.Google ScholarGoogle ScholarCross RefCross Ref
  22. Danny Dolev and Rudiger Reischuk. 1985. Bounds on information exchange for Byzantine agreement. JACM (1985).Google ScholarGoogle Scholar
  23. Sisi Duan, Michael K. Reiter, and Haibin Zhang. 2018. BEAT: Asynchronous BFT Made Practical. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15--19, 2018, David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.). ACM, 2028--2041.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Bryan Ford. 2019. Threshold logical clocks for asynchronous distributed coordination and consensus. arXiv preprint arXiv:1907.07010 (2019).Google ScholarGoogle Scholar
  25. Juan A. Garay, Aggelos Kiayias, and Nikos Leonardos. 2015. The Bitcoin Backbone Protocol: Analysis and Applications. In Advances in Cryptology - EUROCRYPT 2015 - 34th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Sofia, Bulgaria, April 26--30, 2015, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 9057), Elisabeth Oswald and Marc Fischlin (Eds.). Springer, 281--310. Google ScholarGoogle ScholarCross RefCross Ref
  26. Bingyong Guo, Zhenliang Lu, Qiang Tang, Jing Xu, and Zhenfeng Zhang. 2020. Dumbo: Faster Asynchronous BFT Protocols. In CCS '20: 2020 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, USA, November 9--13, 2020, Jay Ligatti, Xinming Ou, Jonathan Katz, and Giovanni Vigna (Eds.). ACM, 803--818.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Runchao Han, Gary Shapiro, Vincent Gramoli, and Xiwei Xu. 2020. On the performance of distributed ledgers for internet of things. Internet of Things 10 (2020), 100087.Google ScholarGoogle ScholarCross RefCross Ref
  28. Idit Keidar, Eleftherios Kokoris-Kogias, Oded Naor, and Alexander Spiegelman. 2021. All You Need is DAG. In Proceedings of the 40th Symposium on Principles of Distributed Computing (Virtual Event, Italy) (PODC '21). Association for Computing Machinery, New York, NY, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Eleftherios Kokoris Kogias, Philipp Jovanovic, Nicolas Gailly, Ismail Khoffi, Linus Gasser, and Bryan Ford. 2016. Enhancing Bitcoin Security and Performance with Strong Consistency via Collective Signing. In 25th USENIX Security Symposium (USENIX Security 16). USENIX Association, Austin, TX, 279--296. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/kogiasGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  30. Eleftherios Kokoris-Kogias, Philipp Jovanovic, Linus Gasser, Nicolas Gailly, Ewa Syta, and Bryan Ford. 2018. OmniLedger: A Secure, Scale-Out, Decentralized Ledger via Sharding. In 2018 IEEE Symposium on Security and Privacy, SP 2018, Proceedings, 21--23 May 2018, San Francisco, California, USA. IEEE Computer Society, 583--598.Google ScholarGoogle ScholarCross RefCross Ref
  31. Eleftherios Kokoris-Kogias, Dahlia Malkhi, and Alexander Spiegelman. 2020. Asynchronous Distributed Key Generation for Computationally-Secure Randomness, Consensus, and Threshold Signatures. In CCS '20: 2020 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, USA, November 9--13, 2020, Jay Ligatti, Xinming Ou, Jonathan Katz, and Giovanni Vigna (Eds.). ACM, 1751--1767.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Andrew Miller, Yu Xia, Kyle Croman, Elaine Shi, and Dawn Song. 2016. The honey badger of BFT protocols. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 31--42.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Satoshi Nakamoto. 2008. Bitcoin whitepaper.Google ScholarGoogle Scholar
  34. Thanh Son Lam Nguyen, Guillaume Jourjon, Maria Potop-Butucaru, and Kim Loan Thai. 2019. Impact of network delays on Hyperledger Fabric. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, 222--227.Google ScholarGoogle ScholarCross RefCross Ref
  35. A Pinar Ozisik, Gavin Andresen, Brian N Levine, Darren Tapp, George Bissias, and Sunny Katkuri. 2019. Graphene: efficient interactive set reconciliation applied to blockchain propagation. In Proceedings of the ACM Special Interest Group on Data Communication. 303--317.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Alexander Spiegelman, Arik Rinberg, and Dahlia Malkhi. 2020. ACE: Abstract Consensus Encapsulation for Liveness Boosting of State Machine Replication. In OPODIS.Google ScholarGoogle Scholar
  37. Chrysoula Stathakopoulou, Tudor David, and Marko Vukolic. 2019. Mir-BFT: High-Throughput BFT for Blockchains. CoRR abs/1906.05552 (2019). arXiv:1906.05552 http://arxiv.org/abs/1906.05552Google ScholarGoogle Scholar
  38. Maofan Yin, Dahlia Malkhi, Michael K. Reiter, Guy Golan-Gueta, and Ittai Abraham. 2019. HotStuff: BFT Consensus with Linearity and Responsiveness. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, PODC 2019, Toronto, ON, Canada, July 29 - August 2, 2019, Peter Robinson and Faith Ellen (Eds.). ACM, 347--356.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Mahdi Zamani, Mahnush Movahedi, and Mariana Raykova. 2018. RapidChain: Scaling Blockchain via Full Sharding. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15--19, 2018, David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.). ACM, 931--948.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Jiashuo Zhang, Jianbo Gao, Zhenhao Wu, Wentian Yan, Qize Wu, Qingshan Li, and Zhong Chen. 2019. Performance Analysis of the Libra Blockchain: An Experimental Study. CoRR abs/1912.05241 (2019).Google ScholarGoogle Scholar

Index Terms

  1. Narwhal and Tusk: a DAG-based mempool and efficient BFT consensus

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      EuroSys '22: Proceedings of the Seventeenth European Conference on Computer Systems
      March 2022
      783 pages
      ISBN:9781450391627
      DOI:10.1145/3492321

      Copyright © 2022 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 28 March 2022

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate241of1,308submissions,18%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader