Skip to main content

Improving query processing in blockchain systems by using a multi-level sharding mechanism

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

A Publisher Correction to this article was published on 17 April 2024

This article has been updated

Abstract

With the distributed and decentralized nature of blockchain, and with its sequential data access, query processing emerges as a challenging issue in the blockchain systems. These features hinder efficient query processing and make it difficult to guarantee the validity and privacy-preserving of query results. Several solutions have been proposed to tackle the efficiency, reliability, and privacy challenges of query processing in blockchain systems. There has been rarely a comprehensive solution addressing all of these issues. In addition, the existing solutions often assume that the blockchain nodes are homogeneous in terms of their capabilities and available resources, while the blockchain nodes can have heterogeneous computational, communication, and storage resources, and can also contribute to the blockchain network in different manners. This work, considering the heterogeneity of network nodes, introduces a multi-level and score-based sharding solution for query processing where the nodes are organized into a hierarchical tree-like structure based on their score and store a proportion of transaction data in a DAG-based data structure resulting in an efficient query time. Additionally, the nodes reach a consensus over the query results from the bottom to the top of the hierarchical structure enabling reliable and fast query processing. The experiments conducted during the evaluation show that the efficiency of the proposed work is near that of relational databases in terms of query response time. It also provides a high validity rate taking advantage of its hierarchical consensus mechanism and preserves the privacy of query results using a delegation-based integration method where the final query result is integrated by the client’s representative.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Change history

References

  1. Jia D-Y, Xin J-C, Wang Z-Q, Lei H, Wang G-R (2021) SE-chain: a scalable storage and efficient retrieval model for blockchain. J Comput Sci Technol 36(3):693–706

    Article  Google Scholar 

  2. Qu Q, Nurgaliev I, Muzammal M, Jensen CS, Fan J (2019) On spatio-temporal blockchain query processing. Futur Gener Comput Syst 98:208–218. https://doi.org/10.1016/j.future.2019.03.038

    Article  Google Scholar 

  3. Muzammal M, Qu Q, Nasrulin B (2019) Renovating blockchain with distributed databases: an open source system. Futur Gener Comput Syst 90:105–117

    Article  Google Scholar 

  4. Xing X, Chen Y, Li T, Xin Y, Sun H (2021) A blockchain index structure based on subchain query. J Cloud Comput 10:1–11

    Article  Google Scholar 

  5. Gürsoy G, Brannon CM, Gerstein M (2020) Using Ethereum blockchain to store and query pharmacogenomics data via smart contracts. BMC Med Genomics 13(1):1–11

    Article  Google Scholar 

  6. Li D, Han D, Crespi N, Minerva R, Li K-C (2023) A blockchain-based secure storage and access control scheme for supply chain finance. J Supercomput 79(1):109–138

    Article  Google Scholar 

  7. Hao K, Xin J, Wang Z, Yao Z, Wang G (2022) On efficient top-k transaction path query processing in blockchain database. Data Knowl Eng 141:102079

    Article  Google Scholar 

  8. Wang H, Xu C, Zhang C, Xu J, Peng Z, Pei J (2022) vchain+: optimizing verifiable blockchain boolean range queries. In: 2022 IEEE 38th International Conference on Data Engineering (ICDE). pp 1927–1940

  9. Rahman MS, Khalil I, Moustafa N, Kalapaaking AP, Bouras A (2021) A blockchain-enabled privacy-preserving verifiable query framework for securing cloud-assisted industrial internet of things systems. IEEE Trans Ind Informatics 18(7):5007–5017

    Article  Google Scholar 

  10. Jiang S, Liu J, Chen J, Liu Y, Wang L, Zhou Y (2022) Query integrity meets blockchain: a privacy-preserving verification framework for outsourced encrypted data. IEEE Trans Serv Comput 16(3):2100–2113

    Google Scholar 

  11. Yang W, Sun B, Zhu Y, Wu D (2021) A secure heuristic semantic searching scheme with blockchain-based verification. Inf Process Manag 58(4):102548

    Article  Google Scholar 

  12. Ma X, Wang C, Chen X (2021) Trusted data sharing with flexible access control based on blockchain. Comput Stand Interfaces 78:103543

    Article  Google Scholar 

  13. Zou R, Lv X, Zhao J (2021) SPChain: Blockchain-based medical data sharing and privacy-preserving eHealth system. Inf Process Manag 58(4):102604

    Article  Google Scholar 

  14. Abuhashim A, Tan CC (2020) Smart contract designs on blockchain applications. In: 2020 IEEE Symposium on Computers and Communications (ISCC). pp 1–4

  15. Chishti MS, Sufyan F, Banerjee A (2021) Decentralized on-chain data access via smart contracts in ethereum blockchain. IEEE Trans Netw Serv Manag 19(1):174–187

    Article  Google Scholar 

  16. Li Y, Zheng K, Yan Y, Liu Q, Zhou X (2017) EtherQL: a query layer for blockchain system. In: Database Systems for Advanced Applications: 22nd International Conference, DASFAA 2017, Suzhou, China, March 27–30, Proceedings, Part II 22. pp 556–567

  17. Pratama FA, Mutijarsa K, (2018) Query support for data processing and analysis on ethereum blockchain. In: 2018 International Symposium on Electronics and Smart Devices (ISESD). pp 1–5

  18. Wang S, et al. (2018) Forkbase: an efficient storage engine for blockchain and forkable applications. arXiv Prepr. arXiv:1802.04949

  19. Zhang Z, Zhong Y, Yu X (2021) Blockchain storage middleware based on external database. In: 2021 6th International Conference on Intelligent Computing and Signal Processing (ICSP). pp 1301–1304

  20. Zeng L, Qiu W, Wang X, Wang H, Yao Y, Yu Z (2021) Transaction-based static ındexing method to ımprove the efficiency of query on the blockchain. In: 2021 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA). pp 780–784

  21. Huang T-L, Huang J, (2020) An efficient data structure for distributed ledger in blockchain systems. In: 2020 International Computer Symposium (ICS). pp 175–178

  22. Ruan P, Anh Dinh TT, Lin Q, Zhang M, Chen G, Chin Ooi B (2020) Revealing every story of data in blockchain systems. ACM Sigmod Rec 49(1):70–77

    Article  Google Scholar 

  23. Xu Y, Zhao S, Kong L, Zheng Y, Zhang S, Li Q (2017) ECBC: a high performance educational certificate blockchain with efficient query. In: Theoretical Aspects of Computing–ICTAC 2017: 14th International Colloquium, Hanoi, Vietnam, October 23–27, Proceedings 14. pp 288–304

  24. Xu C, Zhang C, Xu J (2019) Vchain: enabling verifiable boolean range queries over blockchain databases. In: Proceedings of the 2019 International Conference on Management of Data. pp 141–158

  25. Zhu Y, Zhang Z, Jin C, Zhou A (2020) Enabling generic verifiable aggregate query on blockchain systems. In: 2020 IEEE 26th International Conference on Parallel and Distributed Systems (ICPADS). pp 456–465

  26. Loporchio M, Bernasconi A, Maesa DDF, Ricci L (2021) Authenticating spatial queries on blockchain systems. IEEE Access 9:163363–163378

    Article  Google Scholar 

  27. Tian G, Wei J, Kutyłowski M, Susilo W, Huang X, Chen X (2022) VRBC: a verifiable redactable blockchain with efficient query and integrity auditing. IEEE Trans Comput 72(7):1928–1942

  28. Peng Z et al (2021) Vfchain: enabling verifiable and auditable federated learning via blockchain systems. IEEE Trans Netw Sci Eng 9(1):173–186

    Article  Google Scholar 

  29. Shao Q, Pang S, Zhang Z, and Jing C (2020) Authenticated range query using SGX for blockchain light clients. In: Database Systems for Advanced Applications: 25th International Conference, DASFAA 2020, Jeju, South Korea, September 24–27, Proceedings, Part III 25. pp 306–321

  30. Pang S, Shao Q, Zhang Z, Jin C (2020) Authqx: enabling authenticated query over blockchain via intel sgx. In: Database Systems for Advanced Applications: 25th International Conference, DASFAA 2020, Jeju, South Korea, September 24–27, Proceedings, Part III 25. pp 727–731

  31. Niu Y, Zhang C, Wei L, Xie Y, Zhang X, Fang Y (2019) An efficient query scheme for privacy-preserving lightweight bitcoin client with Intel SGX. In: 2019 IEEE Global Communications Conference (GLOBECOM). pp 1–6

  32. Zhou W, Cai Y, Peng Y, Wang S, Ma K, Li F (2021) Veridb: an sgx-based verifiable database. In: Proceedings of the 2021 International Conference on Management of Data. pp 2182–2194

  33. Wu H, Peng Z, Guo S, Yang Y, Xiao B (2021) VQL: efficient and verifiable cloud query services for blockchain systems. IEEE Trans Parallel Distrib Syst 33(6):1393–1406

    Article  Google Scholar 

  34. Cai C, Weng J, Yuan X, Wang C (2018) Enabling reliable keyword search in encrypted decentralized storage with fairness. IEEE Trans Dependable Secur Comput 18(1):131–144

    Article  Google Scholar 

  35. Tahir S, Rajarajan M (2018) Privacy-preserving searchable encryption framework for permissioned blockchain networks. In: 2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData). pp 1628–1633

  36. Hu S, Cai C, Wang Q, Wang C, Wang Z, Ye D (2019) Augmenting encrypted search: a decentralized service realization with enforced execution. IEEE Trans Dependable Secur Comput 18(6):2569–2581

    Google Scholar 

  37. Chen C-L, Yang J, Tsaur W-J, Weng W, Wu C-M, Wei X (2022) Enterprise data sharing with privacy-preserved based on hyperledger fabric blockchain in IIOT’s application. Sensors 22(3):1146

    Article  Google Scholar 

  38. Chen Y, Bai J, Hao Y, Liao S, Yi Z, Zhang H, (2020) Blockchain-based dynamic group management for multiple keywords searchable encryption technology. In: 2020 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC). pp 1–6

  39. Linoy S, Mahdikhani H, Ray S, Lu R, Stakhanova N, Ghorbani A (2019) Scalable privacy-preserving query processing over ethereum blockchain. In: Proceedings—2019 2nd IEEE International Conference Blockchain, Blockchain 2019, pp 398–404. https://doi.org/10.1109/Blockchain.2019.00061

  40. Ge L, Jiang T (2021) A privacy protection method of lightweight nodes in blockchain. Secur Commun Networks 2021:1–17

    Article  Google Scholar 

  41. Yang M, Margheri A, Hu R, Sassone V (2018) Differentially private data sharing in a cloud federation with blockchain. IEEE Cloud Comput 5(6):69–79

    Article  Google Scholar 

  42. Zhao Y et al (2021) A blockchain-based approach for saving and tracking differential-privacy cost. IEEE Internet Things J 8(11):8865–8882

    Article  Google Scholar 

  43. Xu L, Bao T, Zhu L (2020) Blockchain empowered differentially private and auditable data publishing in industrial IoT. IEEE Trans Ind Informatics 17(11):7659–7668

    Article  Google Scholar 

  44. Luu L, Narayanan V, Zheng C, Baweja K, Gilbert S, Saxena P (2016) A secure sharding protocol for open blockchains. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. pp 17–30

  45. Zamani M, Movahedi M, Raykova M (2018) Rapidchain: Scaling blockchain via full sharding. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. pp 931–948

  46. Buterin V (2013) Ethereum whitepaper. https://ethereum.org/en/whitepaper/ Accessed 27 Mar 2021

  47. LeMahieu C (2018) Nano: a feeless distributed cryptocurrency network. Nano https://nano.org/en/whitepaper Accessed 24 March 2018

  48. Liu Y, Liu J, Li D, Yu H, Wu Q (2020) Fleetchain: a secure scalable and responsive blockchain achieving optimal sharding. In: International Conference on Algorithms and Architectures for Parallel Processing. pp 409–425

  49. Wang J, Wang H (2019) Monoxide: scale out blockchains with asynchronous consensus zones. In: 16th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 19). pp 95–112

  50. Hong Z, Guo S, Li P, Chen W (2021) Pyramid: a layered sharding blockchain system. In: IEEE INFOCOM 2021-IEEE Conference on Computer Communications. pp 1–10

  51. Huang C et al (2020) RepChain: a reputation-based secure, fast, and high incentive blockchain system via sharding. IEEE Internet Things J 8(6):4291–4304

    Article  Google Scholar 

  52. Wu Z, Liu H, Xie J, Xu G, Li G, Lu C (2023) An effective method for the protection of user health topic privacy for health information services. World Wide Web 26(6):1–23. https://doi.org/10.1007/s11280-023-01208-5

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Contributions

Alemeh Matani was responsible for writing—original draft, investigation, conceptualization, methodology, and validation. Amir Sahafi and Ali Broumandnia took part in review and supervision.

Corresponding author

Correspondence to Amir Sahafi.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Matani, A., Sahafi, A. & Broumandnia, A. Improving query processing in blockchain systems by using a multi-level sharding mechanism. J Supercomput 80, 15066–15096 (2024). https://doi.org/10.1007/s11227-024-06037-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-024-06037-5

Keywords