ABSTRACT
We propose separating the task of reliable transaction dissemination from transaction ordering, to enable high-performance Byzantine fault-tolerant quorum-based consensus. We design and evaluate a mempool protocol, Narwhal, specializing in high-throughput reliable dissemination and storage of causal histories of transactions. Narwhal tolerates an asynchronous network and maintains high performance despite failures. Narwhal is designed to easily scale-out using multiple workers at each validator, and we demonstrate that there is no foreseeable limit to the throughput we can achieve.
Composing Narwhal with a partially synchronous consensus protocol (Narwhal-HotStuff) yields significantly better throughput even in the presence of faults or intermittent loss of liveness due to asynchrony. However, loss of liveness can result in higher latency. To achieve overall good performance when faults occur we design Tusk, a zero-message overhead asynchronous consensus protocol, to work with Narwhal. We demonstrate its high performance under a variety of configurations and faults.
As a summary of results, on a WAN, Narwhal-Hotstuff achieves over 130,000 tx/sec at less than 2-sec latency compared with 1,800 tx/sec at 1-sec latency for Hotstuff. Additional workers increase throughput linearly to 600,000 tx/sec without any latency increase. Tusk achieves 160,000 tx/sec with about 3 seconds latency. Under faults, both protocols maintain high throughput, but Narwhal-HotStuff suffers from increased latency.
- Daniel J Abadi and Jose M Faleiro. 2018. An overview of deterministic database systems. Commun. ACM 61, 9 (2018), 78--88.Google ScholarDigital Library
- Michael Abd-El-Malek, Gregory R. Ganger, Garth R. Goodson, Michael K. Reiter, and Jay J. Wylie. 2005. Fault-scalable Byzantine fault-tolerant services. In Proceedings of the 20th ACM Symposium on Operating Systems Principles 2005, SOSP 2005, Brighton, UK, October 23--26, 2005, Andrew Herbert and Kenneth P. Birman (Eds.). ACM, 59--74.Google ScholarDigital Library
- Ittai Abraham, Dahlia Malkhi, and Alexander Spiegelman. 2019. Asymptotically Optimal Validated Asynchronous Byzantine Agreement. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, PODC 2019, Toronto, ON, Canada, July 29 - August 2, 2019, Peter Robinson and Faith Ellen (Eds.). ACM, 337--346.Google ScholarDigital Library
- Mustafa Al-Bassam, Alberto Sonnino, Shehar Bano, Dave Hrycyszyn, and George Danezis. 2018. Chainspace: A Sharded Smart Contracts Platform. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18--21, 2018. The Internet Society.Google Scholar
- Salem Alqahtani and Murat Demirbas. 2021. Bottlenecks in Blockchain Consensus Protocols. CoRR abs/2103.04234 (2021).Google Scholar
- Yair Amir, Brian Coan, Jonathan Kirsch, and John Lane. 2010. Prime: Byzantine replication under attack. IEEE transactions on dependable and secure computing 8, 4 (2010), 564--577.Google Scholar
- Elli Androulaki, Artem Barger, Vita Bortnikov, Christian Cachin, Konstantinos Christidis, Angelo De Caro, David Enyeart, Christopher Ferris, Gennady Laventman, Yacov Manevich, Srinivasan Muralidharan, Chet Murthy, Binh Nguyen, Manish Sethi, Gari Singh, Keith Smith, Alessandro Sorniotti, Chrysoula Stathakopoulou, Marko Vukolic, Sharon Weed Cocco, and Jason Yellick. 2018. Hyperledger fabric: a distributed operating system for permissioned blockchains. In Proceedings of the Thirteenth EuroSys Conference, EuroSys 2018, Porto, Portugal, April 23--26, 2018, Rui Oliveira, Pascal Felber, and Y. Charlie Hu (Eds.). ACM, 30:1--30:15.Google ScholarDigital Library
- Georgia Avarikioti, Eleftherios Kokoris-Kogias, and Roger Wattenhofer. 2021. Divide and Scale: Formalization of Distributed Ledger Sharding Protocols. arXiv:1910.10434 [cs.DC]Google Scholar
- Vivek Bagaria, Sreeram Kannan, David Tse, Giulia Fanti, and Pramod Viswanath. 2019. Prism: Deconstructing the blockchain to approach physical limits. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 585--602.Google ScholarDigital Library
- Leemon Baird. 2016. The swirlds hashgraph consensus algorithm: Fair, fast, byzantine fault tolerance. Swirlds Tech Reports SWIRLDS-TR-2016-01, Tech. Rep (2016).Google Scholar
- Shehar Bano, Alberto Sonnino, Andrey Chursin, Dmitri Perelman, and Dahlia Malkhi. 2020. Twins: White-Glove Approach for BFT Testing. arXiv preprint arXiv:2004.10617 (2020).Google Scholar
- Mathieu Baudet, Avery Ching, Andrey Chursin, George Danezis, François Garillot, Zekun Li, Dahlia Malkhi, Oded Naor, Dmitri Perelman, and Alberto Sonnino. 2019. State machine replication in the Libra blockchain. The Libra Assn., Tech. Rep (2019).Google Scholar
- Alysson Bessani, Joao Sousa, and Eduardo EP Alchieri. 2014. State machine replication for the masses with BFT-SMART. In 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. IEEE, 355--362.Google ScholarDigital Library
- Dan Boneh, Ben Lynn, and Hovav Shacham. 2004. Short Signatures from the Weil Pairing. J. Cryptol. 17, 4 (2004), 297--319.Google ScholarDigital Library
- Gabriel Bracha. 1987. Asynchronous Byzantine agreement protocols. Information and Computation 75, 2 (1987), 130--143.Google ScholarDigital Library
- Gabriel Bracha and Sam Toueg. 1985. Asynchronous consensus and broadcast protocols. Journal of the ACM (JACM) 32, 4 (1985), 824--840.Google ScholarDigital Library
- Ethan Buchman. 2016. Tendermint: Byzantine fault tolerance in the age of blockchains.Google Scholar
- Christian Cachin, Rachid Guerraoui, and Luís Rodrigues. 2011. Introduction to reliable and secure distributed programming. Springer Science & Business Media.Google Scholar
- Miguel Castro and Barbara Liskov. 2002. Practical byzantine fault tolerance and proactive recovery. ACM Trans. Comput. Syst. 20, 4 (2002), 398--461. Google ScholarDigital Library
- George Danezis and David Hrycyszyn. 2018. Blockmania: from block DAGs to consensus. arXiv preprint arXiv:1809.01620 (2018).Google Scholar
- Danny Dolev, Michael J Fischer, Rob Fowler, Nancy A Lynch, and H Raymond Strong. 1982. An efficient algorithm for Byzantine agreement without authentication. Information and Control 52, 3 (1982), 257--274.Google ScholarCross Ref
- Danny Dolev and Rudiger Reischuk. 1985. Bounds on information exchange for Byzantine agreement. JACM (1985).Google Scholar
- Sisi Duan, Michael K. Reiter, and Haibin Zhang. 2018. BEAT: Asynchronous BFT Made Practical. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15--19, 2018, David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.). ACM, 2028--2041.Google ScholarDigital Library
- Bryan Ford. 2019. Threshold logical clocks for asynchronous distributed coordination and consensus. arXiv preprint arXiv:1907.07010 (2019).Google Scholar
- Juan A. Garay, Aggelos Kiayias, and Nikos Leonardos. 2015. The Bitcoin Backbone Protocol: Analysis and Applications. In Advances in Cryptology - EUROCRYPT 2015 - 34th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Sofia, Bulgaria, April 26--30, 2015, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 9057), Elisabeth Oswald and Marc Fischlin (Eds.). Springer, 281--310. Google ScholarCross Ref
- Bingyong Guo, Zhenliang Lu, Qiang Tang, Jing Xu, and Zhenfeng Zhang. 2020. Dumbo: Faster Asynchronous BFT Protocols. In CCS '20: 2020 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, USA, November 9--13, 2020, Jay Ligatti, Xinming Ou, Jonathan Katz, and Giovanni Vigna (Eds.). ACM, 803--818.Google ScholarDigital Library
- Runchao Han, Gary Shapiro, Vincent Gramoli, and Xiwei Xu. 2020. On the performance of distributed ledgers for internet of things. Internet of Things 10 (2020), 100087.Google ScholarCross Ref
- Idit Keidar, Eleftherios Kokoris-Kogias, Oded Naor, and Alexander Spiegelman. 2021. All You Need is DAG. In Proceedings of the 40th Symposium on Principles of Distributed Computing (Virtual Event, Italy) (PODC '21). Association for Computing Machinery, New York, NY, USA.Google ScholarDigital Library
- Eleftherios Kokoris Kogias, Philipp Jovanovic, Nicolas Gailly, Ismail Khoffi, Linus Gasser, and Bryan Ford. 2016. Enhancing Bitcoin Security and Performance with Strong Consistency via Collective Signing. In 25th USENIX Security Symposium (USENIX Security 16). USENIX Association, Austin, TX, 279--296. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/kogiasGoogle ScholarDigital Library
- Eleftherios Kokoris-Kogias, Philipp Jovanovic, Linus Gasser, Nicolas Gailly, Ewa Syta, and Bryan Ford. 2018. OmniLedger: A Secure, Scale-Out, Decentralized Ledger via Sharding. In 2018 IEEE Symposium on Security and Privacy, SP 2018, Proceedings, 21--23 May 2018, San Francisco, California, USA. IEEE Computer Society, 583--598.Google ScholarCross Ref
- Eleftherios Kokoris-Kogias, Dahlia Malkhi, and Alexander Spiegelman. 2020. Asynchronous Distributed Key Generation for Computationally-Secure Randomness, Consensus, and Threshold Signatures. In CCS '20: 2020 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, USA, November 9--13, 2020, Jay Ligatti, Xinming Ou, Jonathan Katz, and Giovanni Vigna (Eds.). ACM, 1751--1767.Google ScholarDigital Library
- Andrew Miller, Yu Xia, Kyle Croman, Elaine Shi, and Dawn Song. 2016. The honey badger of BFT protocols. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 31--42.Google ScholarDigital Library
- Satoshi Nakamoto. 2008. Bitcoin whitepaper.Google Scholar
- Thanh Son Lam Nguyen, Guillaume Jourjon, Maria Potop-Butucaru, and Kim Loan Thai. 2019. Impact of network delays on Hyperledger Fabric. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, 222--227.Google ScholarCross Ref
- A Pinar Ozisik, Gavin Andresen, Brian N Levine, Darren Tapp, George Bissias, and Sunny Katkuri. 2019. Graphene: efficient interactive set reconciliation applied to blockchain propagation. In Proceedings of the ACM Special Interest Group on Data Communication. 303--317.Google ScholarDigital Library
- Alexander Spiegelman, Arik Rinberg, and Dahlia Malkhi. 2020. ACE: Abstract Consensus Encapsulation for Liveness Boosting of State Machine Replication. In OPODIS.Google Scholar
- Chrysoula Stathakopoulou, Tudor David, and Marko Vukolic. 2019. Mir-BFT: High-Throughput BFT for Blockchains. CoRR abs/1906.05552 (2019). arXiv:1906.05552 http://arxiv.org/abs/1906.05552Google Scholar
- Maofan Yin, Dahlia Malkhi, Michael K. Reiter, Guy Golan-Gueta, and Ittai Abraham. 2019. HotStuff: BFT Consensus with Linearity and Responsiveness. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, PODC 2019, Toronto, ON, Canada, July 29 - August 2, 2019, Peter Robinson and Faith Ellen (Eds.). ACM, 347--356.Google ScholarDigital Library
- Mahdi Zamani, Mahnush Movahedi, and Mariana Raykova. 2018. RapidChain: Scaling Blockchain via Full Sharding. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15--19, 2018, David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.). ACM, 931--948.Google ScholarDigital Library
- Jiashuo Zhang, Jianbo Gao, Zhenhao Wu, Wentian Yan, Qize Wu, Qingshan Li, and Zhong Chen. 2019. Performance Analysis of the Libra Blockchain: An Experimental Study. CoRR abs/1912.05241 (2019).Google Scholar
Index Terms
- Narwhal and Tusk: a DAG-based mempool and efficient BFT consensus
Recommendations
HOME: Heard-of Based Formal Modeling and Verification Environment for Consensus Protocols
ICSE '23: Proceedings of the 45th International Conference on Software Engineering: Companion ProceedingsConsensus protocol plays an important role in ensuring the reliability of distributed systems. How to formally model and verify it is a hot research issue. Due to the limitation of verification performance, it can usually verify consensus algorithms ...
Bullshark: DAG BFT Protocols Made Practical
CCS '22: Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications SecurityWe present Bullshark, the first directed acyclic graph (DAG) based asynchronous Byzantine Atomic Broadcast protocol that is optimized for the common synchronous case. Like previous DAG-based BFT protocols [19, 25], Bullshark requires no extra ...
An Experimental Evaluation of BFT Protocols for Blockchains
Blockchain – ICBC 2019AbstractByzantine Fault Tolerant (BFT) protocols have been used in blockchains due to their high performance and fast block acceptance. However, their weakness is a lack of scalability to support a large number of nodes in the network due to message ...
Comments