skip to main content
10.1145/2851553.2858667acmconferencesArticle/Chapter ViewAbstractPublication PagesicpeConference Proceedingsconference-collections
research-article

BFT-Bench: A Framework to Evaluate BFT Protocols

Published: 12 March 2016 Publication History

Abstract

Byzantine Fault Tolerance (BFT) has been extensively studied and numerous protocols and software prototypes have been proposed. However, most BFT prototypes have been evaluated in an ad-hoc setting, considering different fault types and fault injection scenarios. In this paper, we present BFT-Bench, the first benchmarking framework for evaluating and comparing BFT protocols in practice. BFT-Bench includes different BFT protocols implementations, their automatic deployment in a distributed setting, the ability to define and inject different faulty behaviors, and the online monitoring and reporting of performance and dependability measures. Preliminary results of BFT-Bench show the effectiveness of the framework, easily allowing an empirical comparison of different BFT protocols, in various workload and fault scenarios.

References

[1]
Y. Amir, B. A. Coan, J. Kirsch, and J. Lane. Byzantine Replication Under Attack. In DSN, pages 197--206, 2008.
[2]
P.-L. Aublin, S. B. Mokhtar, and V. Quéma. RBFT: Redundant Byzantine Fault Tolerance. In ICDCS, pages 297--306, 2013.
[3]
J.-P. Bahsoun, R. Guerraoui, and A. Shoker. Making BFT Protocols Adaptive.
[4]
A. Bessani, J. Sousa, and E. E. Alchieri. State Machine Replication for the Masses with BFT-SMART. In Dependable Systems and Networks (DSN), 2014 44th Annual IEEE/IFIP International Conference on, pages 355--362. IEEE, 2014.
[5]
F. Cappello, E. Caron, M. Dayde, F. Desprez, Y. Jégou, P. Primet, E. Jeannot, S. Lanteri, J. Leduc, N. Melab, et al. Grid'5000: A Large Scale and Highly Reconfigurable Grid Experimental Testbed. In Proceedings of the 6th IEEE/ACM International Workshop on Grid Computing, pages 99--106. IEEE Computer Society, 2005.
[6]
M. Castro and B. Liskov. Practical Byzantine Fault Tolerance. In OSDI, pages 173--186, 1999.
[7]
A. Clement, E. L. Wong, L. Alvisi, M. Dahlin, and M. Marchetti. Making Byzantine Fault Tolerant Systems Tolerate Byzantine Faults. In NSDI, pages 153--168, 2009.
[8]
M. Correia, N. F. Neves, and P. Verıssimo. BFT-TO: Intrusion Tolerance with Less Replicas. Comput. J., 56(6):693--715, 2013.
[9]
G. S. V. et. al. Spin one's wheels? byzantine fault tolerance with a spinning primary.
[10]
R. Guerraoui, N. Knezevic, V. Quéma, and M. Vukolic. The Next 700 BFT Protocols. In EuroSys, pages 363--376, 2010.
[11]
R. Guerraoui, N. Knezevic, V. Quema, and M. Vukolic. Stretching BFT. Technical report, Technical Report EPFL-REPORT-149105, EPFL, 2011.
[12]
R. Kotla, L. Alvisi, M. Dahlin, A. Clement, and E. L. Wong. Zyzzyva: Speculative Byzantine Fault Tolerance. ACM Trans. Comput. Syst., 27(4), 2009.
[13]
L. Lamport. The Part-Time Parliament. ACM Trans. Comput. Syst., 16(2):133--169, 1998.
[14]
L. Lamport. Paxos Made Simple. SIGACT News, 32(4):51--58, 2001.
[15]
F. B. Schneider. Implementing Fault-tolerant Services Using the State Machine Approach: A Tutorial. ACM Comput. Surv., 22(4):299--319, Dec. 1990.
[16]
R. van Renesse and F. B. Schneider. Chain Replication for Supporting High Throughput and Availability. In OSDI, pages 91--104, 2004.
[17]
G. S. Veronese, M. Correia, A. N. Bessani, L. C. Lung, and P. Verıssimo. Efficient Byzantine Fault-Tolerance. IEEE Trans. Computers, 62(1):16--30, 2013.

Cited By

View all
  • (2023)uBFT: Microsecond-Scale BFT using Disaggregated MemoryProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3575693.3575732(862-877)Online publication date: 27-Jan-2023
  • (2023)Leaderless consensusJournal of Parallel and Distributed Computing10.1016/j.jpdc.2023.01.009176(95-113)Online publication date: Jun-2023
  • (2020)The Performance of Byzantine Fault Tolerant Blockchains2020 IEEE 19th International Symposium on Network Computing and Applications (NCA)10.1109/NCA51143.2020.9306742(1-8)Online publication date: 24-Nov-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICPE '16: Proceedings of the 7th ACM/SPEC on International Conference on Performance Engineering
March 2016
346 pages
ISBN:9781450340809
DOI:10.1145/2851553
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 March 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. benchmarking
  2. byzantine faults
  3. fault injection
  4. fault tolerance
  5. performance
  6. robustness

Qualifiers

  • Research-article

Funding Sources

  • European Commission
  • INRIA ALADDIN

Conference

ICPE'16

Acceptance Rates

ICPE '16 Paper Acceptance Rate 23 of 74 submissions, 31%;
Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)20
  • Downloads (Last 6 weeks)2
Reflects downloads up to 27 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)uBFT: Microsecond-Scale BFT using Disaggregated MemoryProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3575693.3575732(862-877)Online publication date: 27-Jan-2023
  • (2023)Leaderless consensusJournal of Parallel and Distributed Computing10.1016/j.jpdc.2023.01.009176(95-113)Online publication date: Jun-2023
  • (2020)The Performance of Byzantine Fault Tolerant Blockchains2020 IEEE 19th International Symposium on Network Computing and Applications (NCA)10.1109/NCA51143.2020.9306742(1-8)Online publication date: 24-Nov-2020
  • (2017)Performance and Dependability Evaluation of Distributed Event-based SystemsProceedings of the 8th ACM/SPEC on International Conference on Performance Engineering10.1145/3030207.3030245(349-352)Online publication date: 17-Apr-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media