research-article

Public Access

Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains

Authors:

Sreeram Kannan,

Pramod ViswanathAuthors Info & Claims

Proceedings of the ACM on Measurement and Analysis of Computing Systems, Volume 6, Issue 1

Article No.: 11, Pages 1 - 38

https://doi.org/10.1145/3508031

Published: 28 February 2022 Publication History

Abstract

In this paper, we study a canonical distributed resource allocation problem arising in blockchains. While distributed resource allocation is a well-studied problem in networking, the blockchain setting additionally requires the solution to be resilient to adversarial behavior from a fraction of nodes. Scaling blockchain performance is a basic research topic; a plethora of solutions (under the umbrella of sharding ) have been proposed in recent years. Although the various sharding solutions share a common thread (they cryptographically stitch together multiple parallel chains), architectural differences lead to differing resource allocation problems. In this paper we make three main contributions: (a) we categorize the different sharding proposals under a common architectural framework, allowing for the emergence of a new, uniformly improved, uni-consensus sharding architecture. (b) We formulate and exactly solve a core resource allocation problem in the uni-consensus sharding architecture -- our solution, Free2shard, is adversary-resistant and achieves optimal throughput. The key technical contribution is a mathematical connection to the classical work of Blackwell approachability in dynamic game theory. (c) We implement the sharding architecture atop a full-stack blockchain in 3000 lines of code in Rust -- we achieve a throughput of more than 250,000 transactions per second with 6 shards, a vast improvement over state-of-the-art.

References

[1]

Jacob Abernethy, Peter L Bartlett, and Elad Hazan. 2011. Blackwell approachability and no-regret learning are equivalent. In Proceedings of the 24th Annual Conference on Learning Theory. JMLR Workshop and Conference Proceedings, 27--46.

[2]

Ittai Abraham, TH Hubert Chan, Danny Dolev, Kartik Nayak, Rafael Pass, Ling Ren, and Elaine Shi. 2019. Communication complexity of byzantine agreement, revisited. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing . 317--326.

Digital Library

[3]

Mustafa Al-Bassam. 2019. LazyLedger: A Distributed Data Availability Ledger With Client-Side Smart Contracts. arxiv: 1905.09274 [cs.CR]

[4]

Bowen Alpern and Fred B Schneider. 1987. Recognizing safety and liveness. Distributed computing, Vol. 2, 3 (1987), 117--126.

[5]

Eitan Altman, Alexandre Reiffers, Daniel S Menasche, Mandar Datar, Swapnil Dhamal, and Corinne Touati. 2019. Mining competition in a multi-cryptocurrency ecosystem at the network edge: A congestion game approach. ACM SIGMETRICS Performance Evaluation Review, Vol. 46, 3 (2019), 114--117.

Digital Library

[6]

Sanjeev Arora, Elad Hazan, and Satyen Kale. 2012. The multiplicative weights update method: a meta-algorithm and applications. Theory of Computing, Vol. 8, 1 (2012), 121--164.

[7]

Georgia Avarikioti, Eleftherios Kokoris-Kogias, and Roger Wattenhofer. 2019. Divide and scale: Formalization of distributed ledger sharding protocols. arXiv preprint arXiv:1910.10434 (2019).

[8]

Christian Badertscher, Juan Garay, Ueli Maurer, Daniel Tschudi, and Vassilis Zikas. 2018a. But why does it work? A rational protocol design treatment of bitcoin. In Annual international conference on the theory and applications of cryptographic techniques . Springer, 34--65.

[9]

Christian Badertscher, Peter Gazi, Aggelos Kiayias, Alexander Russell, and Vassilis Zikas. 2018b. Ouroboros Genesis: Composable Proof-of-Stake Blockchains with Dynamic Availability. Cryptology ePrint Archive, Report 2018/378. https://eprint.iacr.org/2018/378.

[10]

Vivek Bagaria, Sreeram Kannan, David Tse, Giulia Fanti, and Pramod Viswanath. 2019. sf Prism: Deconstructing the blockchain to approach physical limits. ACM Computer and Communications Security Conference (2019).

[11]

Mahesh Balakrishnan, Dahlia Malkhi, Vijayan Prabhakaran, Ted Wobbler, Michael Wei, and John D Davis. 2012. CORFU: A Shared Log Design for Flash Clusters. In Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12) . 1--14.

[12]

Mahesh Balakrishnan, Dahlia Malkhi, Ted Wobber, Ming Wu, Vijayan Prabhakaran, Michael Wei, John D Davis, Sriram Rao, Tao Zou, and Aviad Zuck. 2013. Tango: Distributed data structures over a shared log. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. 325--340.

Digital Library

[13]

Eli Ben-Sasson, Iddo Bentov, Yinon Horesh, and Michael Riabzev. 2018. Scalable, transparent, and post-quantum secure computational integrity. IACR Cryptology ePrint Archive, Vol. 2018 (2018), 46.

[14]

Eli Ben-Sasson, Alessandro Chiesa, Eran Tromer, and Madars Virza. 2014. Succinct non-interactive zero knowledge for a von Neumann architecture. In 23rd USENIX Security Symposium (USENIX Security 14). 781--796.

[15]

David Blackwell et almbox. 1956. An analog of the minimax theorem for vector payoffs. Pacific J. Math., Vol. 6, 1 (1956), 1--8.

[16]

Miguel Castro, Barbara Liskov, et almbox. 1999. Practical Byzantine fault tolerance. In OSDI, Vol. 99. 173--186.

Digital Library

[17]

Danny Dolev and H. Raymond Strong. 1983. Authenticated algorithms for Byzantine agreement. SIAM J. Comput., Vol. 12, 4 (1983), 656--666.

Digital Library

[18]

Cynthia Dwork, Nancy Lynch, and Larry Stockmeyer. 1988. Consensus in the presence of partial synchrony. Journal of the ACM (JACM), Vol. 35, 2 (1988), 288--323.

Digital Library

[19]

Juan Garay, Aggelos Kiayias, and Nikos Leonardos. 2015. The bitcoin backbone protocol: Analysis and applications. In Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer.

[20]

Juan Garay, Aggelos Kiayias, and Nikos Leonardos. 2020. Full Analysis of Nakamoto Consensus in Bounded-Delay Networks. Cryptology ePrint Archive, Report 2020/277. https://eprint.iacr.org/2020/277.

[21]

Juan A. Garay, Aggelos Kiayias, and Nikos Leonardos. 2016. The Bitcoin Backbone Protocol with Chains of Variable Difficulty. Cryptology ePrint Archive, Report 2016/1048. https://eprint.iacr.org/2016/1048.

[22]

Adem Efe Gencer, Robbert van Renesse, and Emin Gün Sirer. 2017. Short paper: Service-oriented sharding for blockchains. In International Conference on Financial Cryptography and Data Security. Springer, 393--401.

[23]

Dileep Kalathil, Vivek S Borkar, and Rahul Jain. 2017. Approachability in Stackelberg stochastic games with vector costs. Dynamic games and applications, Vol. 7, 3 (2017), 422--442.

[24]

Harry Kalodner, Steven Goldfeder, Xiaoqi Chen, S Matthew Weinberg, and Edward W Felten. 2018. Arbitrum: Scalable, private smart contracts. In 27th $$USENIX$$ Security Symposium ($$USENIX$$ Security 18). 1353--1370.

[25]

Aggelos Kiayias, Alexander Russell, Bernardo David, and Roman Oliynykov. 2017. Ouroboros: A provably secure proof-of-stake blockchain protocol. In Annual International Cryptology Conference. Springer, 357--388.

[26]

Eleftherios Kokoris-Kogias, Philipp Jovanovic, Linus Gasser, Nicolas Gailly, Ewa Syta, and Bryan Ford. 2018. Omniledger: A secure, scale-out, decentralized ledger via sharding. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 583--598.

[27]

Leslie Lamport. 1977. Proving the correctness of multiprocess programs. IEEE transactions on software engineering 2 (1977), 125--143.

Digital Library

[28]

Andrew Lewis-Pye and Tim Roughgarden. 2020. Resource Pools and the CAP Theorem. arxiv: 2006.10698 [cs.DC]

[29]

Nikolaos Liakopoulos, Apostolos Destounis, Georgios Paschos, Thrasyvoulos Spyropoulos, and Panayotis Mertikopoulos. 2019. Cautious regret minimization: Online optimization with long-term budget constraints. In International Conference on Machine Learning . PMLR, 3944--3952.

[30]

Loi Luu, Viswesh Narayanan, Chaodong Zheng, Kunal Baweja, Seth Gilbert, and Prateek Saxena. 2016. A secure sharding protocol for open blockchains. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 17--30.

Digital Library

[31]

Loi Luu and Nate Rush. 2017. PeaceRelay Merkle-Patricia Trie Proof Verification.

[32]

Shie Mannor, John N Tsitsiklis, and Jia Yuan Yu. 2009. Online Learning with Sample Path Constraints. Journal of Machine Learning Research, Vol. 10, 3 (2009).

[33]

Satoshi Nakamoto. 2008. Bitcoin: A peer-to-peer electronic cash system. (2008).

[34]

Bryan Parno, Jon Howell, Craig Gentry, and Mariana Raykova. 2013. Pinocchio: Nearly practical verifiable computation. In 2013 IEEE Symposium on Security and Privacy. IEEE, 238--252.

Digital Library

[35]

R. Pass and E. Shi. 2017. Fruitchains: A fair blockchain. In Proceedings of the ACM Symposium on Principles of Distributed Computing. ACM.

[36]

Ethereum Research. 2020. Ethereum 2.0. https://github.com/ethereum/eth2.0-specs .

[37]

Fred B Schneider. 1993. Replication management using the state-machine approach, distributed systems. (1993).

[38]

Nahum Shimkin. 2016. An online convex optimization approach to Blackwell's approachability. The Journal of Machine Learning Research, Vol. 17, 1 (2016), 4434--4456.

Digital Library

[39]

Alex Skidanov and Illia Polosukhin. 2019. Nightshade: Near protocol sharding design. (2019).

[40]

Alberto Sonnino, Shehar Bano, Mustafa Al-Bassam, and George Danezis. 2019. Replay Attacks and Defenses Against Cross-shard Consensus in Sharded Distributed Ledgers. arXiv preprint arXiv:1901.11218 (2019).

[41]

Rayadurgam Srikant and Lei Ying. 2013. Communication networks: an optimization, control, and stochastic networks perspective .Cambridge University Press.

[42]

Leandros Tassiulas and Anthony Ephremides. 1990. Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks. In 29th IEEE Conference on Decision and Control. IEEE, 2130--2132.

[43]

Coin Telegraph. 2019. 10,000 Nodes Are Running BTC Lightning Network in New All-Time High. https://cointelegraph.com/news/10-000-nodes-are-running-btc-lightning-network-in-new-all-time-high .

[44]

Jason Teutsch and Christian Reitwießner. 2019. A scalable verification solution for blockchains. arXiv preprint arXiv:1908.04756 (2019).

[45]

John Von Neumann. 1953. A certain zero-sum two-person game equivalent to the optimal assignment problem. Contributions to the Theory of Games, Vol. 2, 0 (1953), 5--12.

[46]

Heinrich von Stackelberg. 2011. Market Structure and Equilibrium .Springer.

[47]

Jiaping Wang and Hao Wang. 2019. Monoxide: Scale out blockchains with asynchronous consensus zones. In 16th $$USENIX$$ Symposium on Networked Systems Design and Implementation ($$NSDI$$ 19) . 95--112.

[48]

Gavin Wood. 2016. Polkadot: Vision for a heterogeneous multi-chain framework. (2016).

[49]

Lei Yang, Vivek Bagaria, Gerui Wang, Mohammad Alizadeh, David Tse, Giulia Fanti, and Pramod Viswanath. 2019. Prism: Scaling Bitcoin by 10,000 x. arXiv preprint arXiv:1909.11261 (2019).

[50]

Hao Yu and Michael J Neely. 2016. A Low Complexity Algorithm with $O(backslashsqrt T)$ Regret and $O(1)$ Constraint Violations for Online Convex Optimization with Long Term Constraints. arXiv preprint arXiv:1604.02218 (2016).

[51]

Hao Yu, Michael J Neely, and Xiaohan Wei. 2017. Online Convex Optimization with Stochastic Constraints. In 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.

[52]

Mingchao Yu, Saeid Sahraei, Songze Li, Salman Avestimehr, Sreeram Kannan, and Pramod Viswanath. 2019. Coded Merkle Tree: Solving Data Availability Attacks in Blockchains. Cryptology ePrint Archive, Report 2019/1139. https://eprint.iacr.org/2019/1139.

[53]

Mahdi Zamani, Mahnush Movahedi, and Mariana Raykova. 2018. Rapidchain: Scaling blockchain via full sharding. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM.

Digital Library

[54]

Zilliqa. 2017. The ZILLIQA Technical Whitepaper. https://docs.zilliqa.com/whitepaper.pdf .

Cited By

Hu DWang JLiu XLi QLi K(2024)LMChain: An Efficient Load-Migratable Beacon-Based Sharding Blockchain SystemIEEE Transactions on Computers10.1109/TC.2024.340405773:9(2178-2191)Online publication date: Sep-2024
https://doi.org/10.1109/TC.2024.3404057
Xiao GSamian NSeah WHafez Bin Ninggal MBinti Hussin MSahni J(2024)Correlation and Workload-Based Transaction Allocation Algorithm for Blockchain Sharding2024 IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA)10.1109/ISPA63168.2024.00268(1968-1973)Online publication date: 30-Oct-2024
https://doi.org/10.1109/ISPA63168.2024.00268
Li MLin YZhang JWang W(2023)CoChain: High Concurrency Blockchain Sharding via Consensus on ConsensusIEEE INFOCOM 2023 - IEEE Conference on Computer Communications10.1109/INFOCOM53939.2023.10228892(1-10)Online publication date: 17-May-2023
https://doi.org/10.1109/INFOCOM53939.2023.10228892
Show More Cited By

Index Terms

Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains

Recommendations

Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains
SIGMETRICS/PERFORMANCE '22: Abstract Proceedings of the 2022 ACM SIGMETRICS/IFIP PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems

In this paper, we formulate and study a new, but basic, distributed resource allocation problem arising in scaling blockchain performance. While distributed resource allocation is a well-studied problem in networking, the blockchain setting additionally ...
Free2Shard: Adversary-resistant Distributed Resource Allocation for Blockchains
SIGMETRICS '22

In this paper, we formulate and study a new, but basic, distributed resource allocation problem arising in scaling blockchain performance. While distributed resource allocation is a well-studied problem in networking, the blockchain setting additionally ...
Load Balancing for Sharded Blockchains
Financial Cryptography and Data Security
Abstract
Sharding is an approach to designing a highly scalable blockchain. A sharded blockchain achieves parallelism by dividing consensus nodes (validators) into groups called shards and making them process different transactions in each shard. In this ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Measurement and Analysis of Computing Systems

Proceedings of the ACM on Measurement and Analysis of Computing Systems Volume 6, Issue 1

POMACS

March 2022

695 pages

EISSN:2476-1249

DOI:10.1145/3522731

Editors:
Augustin Chaintreau
Columbia University
,
Leana Golubchik
University of Southern California
,
Zhi-Li Zhang
University of Minnesota

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 February 2022

Published in POMACS Volume 6, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation
US Army Research Office Grant
National Science Foundation
XDC network

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
525
Total Downloads

Downloads (Last 12 months)207
Downloads (Last 6 weeks)35

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hu DWang JLiu XLi QLi K(2024)LMChain: An Efficient Load-Migratable Beacon-Based Sharding Blockchain SystemIEEE Transactions on Computers10.1109/TC.2024.340405773:9(2178-2191)Online publication date: Sep-2024
https://doi.org/10.1109/TC.2024.3404057
Xiao GSamian NSeah WHafez Bin Ninggal MBinti Hussin MSahni J(2024)Correlation and Workload-Based Transaction Allocation Algorithm for Blockchain Sharding2024 IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA)10.1109/ISPA63168.2024.00268(1968-1973)Online publication date: 30-Oct-2024
https://doi.org/10.1109/ISPA63168.2024.00268
Li MLin YZhang JWang W(2023)CoChain: High Concurrency Blockchain Sharding via Consensus on ConsensusIEEE INFOCOM 2023 - IEEE Conference on Computer Communications10.1109/INFOCOM53939.2023.10228892(1-10)Online publication date: 17-May-2023
https://doi.org/10.1109/INFOCOM53939.2023.10228892
Pigaglio MKról MRiviére E(2023)Exploring Locality in Ethereum Transactions2023 5th Conference on Blockchain Research & Applications for Innovative Networks and Services (BRAINS)10.1109/BRAINS59668.2023.10317054(1-8)Online publication date: 11-Oct-2023
https://doi.org/10.1109/BRAINS59668.2023.10317054
Liu XXie HYan ZLiang X(2023)A survey on blockchain shardingISA Transactions10.1016/j.isatra.2023.06.029141(30-43)Online publication date: Oct-2023
https://doi.org/10.1016/j.isatra.2023.06.029
David BMagri BMatt CNielsen JTschudi DYin HStavrou ACremers CShi E(2022)GearBoxProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security10.1145/3548606.3559375(683-696)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3548606.3559375

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents