GFBE: A Generalized and Fine-Grained Blockchain Evaluation Framework | IEEE Journals & Magazine | IEEE Xplore

GFBE: A Generalized and Fine-Grained Blockchain Evaluation Framework


Abstract:

Multi-dimensional performance evaluation is crucial for blockchain systems as it enables appropriate blockchain choosing for a given scenario and helps to pinpoint the bo...Show More

Abstract:

Multi-dimensional performance evaluation is crucial for blockchain systems as it enables appropriate blockchain choosing for a given scenario and helps to pinpoint the bottleneck module of a blockchain system to optimize its performance. However, the existing evaluation frameworks for blockchain suffer from low system generality, inefficient workload execution, and incomprehensible evaluation metrics. In order to overcome their limitations, we design and implement the Generalized and Fine-grained Blockchain Evaluation (GFBE) framework. Specifically, we abstract 3 types of Universal Evaluation Interface (UEI) via the dynamic proxying approach to enable generalized evaluation of heterogeneous blockchain systems. Through the design of Lua-based workloads plugin with high flexibility and reusability, GFBE improves the efficiency of workload execution. To achieve comprehensive measurement, we define 15 key performance metrics across hierarchical layers of blockchain architecture. We also implement and deploy GFBE on 16 machines each with 8 CPUs and 16GB RAM, and evaluate three open-source blockchain systems namely Ethereum, ChainMaker, and Haihe smart chain. The experimental results demonstrate that GFBE efficiently and accurately measure 15 key performance metrics such as Contract Execution Efficiency at the contract layer, Consensus Agreement Time Ratio at the consensus layer, and State Query Time at the data layer. Compared with state-of-the-art frameworks such as BLOCKBENCH, Log-based, and Caliper, GFBE distinguishes itself as the only framework that encompasses the appealing features of universal interface, reusable workload, and all-layer metrics.
Published in: IEEE Transactions on Computers ( Volume: 73, Issue: 3, March 2024)
Page(s): 942 - 955
Date of Publication: 08 January 2024

ISSN Information:

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.