Skip to main content
SpringerLink
Log in

On-Chain and Off-Chain Data Management for Blockchain-Internet of Things: A Multi-Agent Deep Reinforcement Learning Approach

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

The emergence of blockchain technology has seen applications increasingly hybridise cloud storage and distributed ledger technology in the Internet of Things (IoT) and cyber-physical systems, complicating data management in decentralised applications (DApps). Because it is inefficient for blockchain technology to handle large amounts of data, effective on-chain and off-chain data management in peer-to-peer networks and cloud storage has drawn considerable attention. Space reservation is a cost-effective approach to managing cloud storage effectively, contrasting with the demand for additional space in real-time. Furthermore, off-chain data replication in the peer-to-peer network can eliminate single points of failure of DApps. However, recent research has rarely discussed optimising on-chain and off-chain data management in the blockchain-enabled IoT (BIoT) environment. In this study, the BIoT environment is modelled, with cloud storage and blockchain orchestrated over the peer-to-peer network. The asynchronous advantage actor-critic algorithm is applied to exploit intelligent agents with the optimal policy for data packing, space reservation, and data replication to achieve an intelligent data management strategy. The experimental analysis reveals that the proposed scheme demonstrates rapid convergence and superior performance in terms of average total reward compared with other typical schemes, resulting in enhanced scalability, security and reliability of blockchain-IoT networks, leading to an intelligent data management strategy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The data analysed in this study are available upon reasonable request.

References

  1. Viriyasitavat, W., Da Xu, L., Bi, Z., Pungpapong, V.: Blockchain and internet of things for modern business process in digital economy—the state of the art. IEEE Trans. Comput. Soc. Syst. 6(6), 1420–1432 (2019)

    Article  Google Scholar 

  2. Novo, O.: Blockchain meets iot: an architecture for scalable access management in iot. IEEE Internet Things J. 5(2), 1184–1195 (2018)

    Article  Google Scholar 

  3. Reyna, A., Martn, C., Chen, J., Soler, E., Daz, M.: On blockchain and its integration with iot. challenges and opportunities. Future Gener. Comput. Syst. 88, 173–190 (2018)

    Article  Google Scholar 

  4. Li, Q.K., Lin, H., Tan, X., Du, S.: H∞ consensus for multiagent-based supply chain systems under switching topology and uncertain demands. IEEE Trans. Syst. Man Cybern.: Syst. 50(12), 4905–4918 (2018)

    Article  Google Scholar 

  5. Wang, S., Sheng, H., Zhang, Y., Yang, D., Shen, J., Chen, R.: Blockchain-empowered distributed multi-camera multi-target tracking in edge computing. IEEE Trans. Industr. Inf. 20(1), 369–379 (2023)

  6. Guo, Y., Zhang, C., Wang, C., Jia, X.: Towards public verifiable and forward-privacy encrypted search by using blockchain. IEEE Trans Dependable Secure Comput. 20(3), 2111–2126 (2022)

  7. Ben-Yair: Updating google photos’ storage policy to build for the future. Google (2020). https://blog.google/products/photos/storage-changes/. Accessed 30 May 2023

  8. Kulshrestha, S., Patel, S.: An efficient host overload detection algorithm for cloud data center based on exponential weighted moving average. Int. J. Commun Syst 34(4), e4708 (2021)

    Article  Google Scholar 

  9. Patel, M., Chaudhary, S., Garg, S.: Machine learning based statistical prediction model for improving performance of live virtual machine migration. J. Eng. 2016, 3061674 (2016)

  10. Bala, A., Chana, I.: Prediction-based proactive load balancing approach through vm migration. Eng. Comput. 32(4), 581–592 (2016)

    Article  Google Scholar 

  11. Saxena, D., Singh, A.K., Buyya, R.: Op-mlb: An online Vm prediction based multi-objective load balancing framework for resource management at cloud datacenter. IEEE Trans. Cloud Comput. 10(4), 2804–2816 (2021)

  12. Zheng, X.R., Lu, Y.: Blockchain technology–recent research and future trend. Enterp. Inf. Syst. 16(12), 1939895 (2022)

    Article  Google Scholar 

  13. Golightly, L., Chang, V., Xu, Q.A., Gao, X., Liu, B.S.: Adoption of cloud computing as innovation in the organisation. Int. J. Eng. Bus. Manag. 14, 18479790221093990 (2022)

    Article  Google Scholar 

  14. Djenouri, Y., Srivastava, G., Belhadi, A., Lin, J.C.W: Intelligent blockchain management for distributed knowledge graphs in IoT 5G environments. Trans. Emerg. Telecommun. Technol. e4332 (2021). https://doi.org/10.1002/ett.4332

  15. Djenouri, Y., Yazidi, A., Srivastava, G., Lin, J.C.W.: Blockchain: applications, challenges, and opportunities in consumer electronics. IEEE Consum. Electron. Mag. (2023). https://doi.org/10.1109/MCE.2023.3247911

  16. Ma, C.Y., Mo, D.Y.: Integrating internet of things in service parts operations for sustainability. Int. J. Eng. Bus. Manag. 15, 18479790231165640 (2023)

    Article  Google Scholar 

  17. Ni, S., Bai, X., Liang, Y., Pang, Z., Li, L.: Blockchain-based traceability system for supply chain: potentials, gaps, applicability and adoption game. Enterp. Inf. Syst. 16(12), 2086021 (2022)

    Article  Google Scholar 

  18. Marchesi, L., Marchesi, M., Tonelli, R.: Abcde—agile block chain dapp engineering. Blockchain: Res. Appl. 1(1–2), 100002 (2020)

    Google Scholar 

  19. Song, W., Xiao, Z., Chen, Qi., Luo, H.: Adaptive resource provisioning for the cloud using online bin packing. IEEE Trans. Comput. 63(11), 2647–2660 (2013)

    Article  MathSciNet  Google Scholar 

  20. Cao, B., Wang, X., Zhang, W., Song, H., Lv, Z.: A many-objective optimisation model of industrial internet of things based on private blockchain. IEEE Netw. 34(5), 78–83 (2020)

    Article  Google Scholar 

  21. Li, K., Ji, L., Yang, S., Li, H., Liao, X.: Couple-group consensus of cooperative–competitive heterogeneous multiagent systems: a fully distributed event-triggered and pinning control method. IEEE Trans. Cybern. 52(6), 4907–4915 (2020)

    Article  Google Scholar 

  22. Bein, D., Bein, W., Venigella, S.: Cloud storage and online bin packing. In: Brazier, F.M.T., Nieuwenhuis, K., Pavlin, G., Warnier, M., Badica, C. (eds.) Intelligent Distributed Computing V. Studies in Computational Intelligence, vol 382, pp. 63–68. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24013-3_7

  23. Mohiuddin, I., Almogren, A., Al Qurishi, M., Hassan, M.M., Al Rassan, I., Fortino, G.: Secure distributed adaptive bin packing algorithm for cloud storage. Future Gener. Comput. Syst. 90, 307–316 (2019)

    Article  Google Scholar 

  24. Woodman, S., Hiden, H., Watson, P.: Applications of provenance in performance prediction and data storage optimisation. Futur. Gener. Comput. Syst. 75, 299–309 (2017)

    Article  Google Scholar 

  25. Ferrer, A.J., Marquès, J.M., Jorba, J.: Towards the decentralised cloud: survey on approaches and challenges for mobile, ad hoc, and edge computing. ACM Comput. Surv. 51(6), 1–36 (2019)

    Article  Google Scholar 

  26. Alam, M.S., Mark, J.W., Shen, X.S.: Relay selection and resource allocation for multi-user cooperative ofdma networks. IEEE Trans. Wirel. Commun. 12(5), 2193–2205 (2013)

    Article  Google Scholar 

  27. Peng, M., Zhang, K., Jiang, J., Wang, J., Wang, W.: Energy-efficient resource assignment and power allocation in heterogeneous cloud radio access networks. IEEE Trans. Veh. Technol. 64(11), 5275–5287 (2014)

    Article  Google Scholar 

  28. Wang, Y., Sheng, M., Wang, X., Wang, L., Li, J.: Mobile-edge computing: Partial computation offloading using dynamic voltage scaling. IEEE Trans. Commun. 64(10), 4268–4282 (2016)

    Google Scholar 

  29. Burd, T.D., Brodersen, R.W.: Processor design for portable systems. J. VLSI Sign. Process Syst. Signal Image Video Technol. 13(2), 203–221 (1996)

    Article  Google Scholar 

  30. Antshares digital assets for everyone [online]. (2016). Available: https://www.antshares.org. Accessed 30 May 2023

  31. Coelho, I.M., Coelho, V.N., Araujo, R.P., Qiang, W.Y., Rhodes, B.D.: Challenges of pbft-inspired consensus for blockchain and enhancements over neo dbft. Future Internet 12(8), 129 (2020)

    Article  Google Scholar 

  32. Liu, M., Yu, F.R., Teng, Y., Leung, V.C.M., Song, M.: Performance optimisation for blockchain-enabled industrial internet of things (iiot) systems: a deep reinforcement learning approach. IEEE Trans. Ind. Inf. 15(6), 3559–3570 (2019)

    Article  Google Scholar 

  33. Clement, A., Wong, E.L., Alvisi, L., Dahlin, M., Marchetti, M.: Making byzantine fault tolerant systems tolerate byzantine faults. In: NSDI, vol 9, pp. 153–168 (2009)

  34. Gomaa, H., Messier, G.G., Williamson, C., Davies, R.: Estimating instantaneous cache hit ratio using markov chain analysis. IEEE/ACM Trans. Netw. 21(5), 1472–1483 (2012)

    Article  Google Scholar 

  35. Breslau, L., Cao, P., Li, F., Phillips, G., Shenker, S.: Web caching and zipf-like distributions: evidence and implications. In: IEEE INFOCOM’99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No. 99CH36320), IEEE, vol 1, pp 126–134 (1999)

  36. Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937. PMLR (2016)

  37. Babaeizadeh, M., Frosio, I., Tyree, S., Clemons, J., Kautz, J.: Reinforcement learning through asynchronous advantage actor-critic on a gpu. arXiv preprint arXiv:1611.06256 (2016)

  38. Zhao, S., Gong, M., Liu, T., Huan, Fu., Tao, D.: Domain generalisation via entropy regularisation. Adv. Neural. Inf. Process. Syst. 33, 16096–16107 (2020)

    Google Scholar 

  39. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)

  40. Belhadi, A., Djenouri, Y., Srivastava, G., Jolfaei, A., Lin, J.C.W.: Privacy reinforcement learning for faults detection in the smart grid. Ad Hoc Netw. 119, 102541 (2021)

    Article  Google Scholar 

  41. Chen, W., Chen, Y., Chen, X., Zheng, Z.: Toward secure data sharing for the iov: a quality-driven incentive mechanism with on-chain and off-chain guarantees. IEEE Internet Things J. 7(3), 1625–1640 (2019)

    Article  MathSciNet  Google Scholar 

  42. Feng, J., Yu, F.R., Pei, Q., Chu, X., Du, J., Zhu, L.: Cooperative computation offloading and resource allocation for blockchain-enabled mobile-edge computing: a deep reinforcement learning approach. IEEE Internet Things J. 7(7), 6214–6228 (2019)

    Article  Google Scholar 

  43. Liu, A., Zhao, D., Li, T.: A data classification method based on particle swarm optimisation and kernel function extreme learning machine. Enterp. Inf. Syst. 17(3), 1913764 (2023)

    Article  Google Scholar 

  44. Wan, H.C., Chin, K.S.: Exploring internet of healthcare things for establishing an integrated care link system in the healthcare industry. Int. J. Eng. Bus. Manag. 13, 18479790211019530 (2021)

    Article  Google Scholar 

  45. Wang, J.W., Ip, W.H., Muddada, R.R., Huang, J.L., Zhang, W.J.: On Petri net implementation of proactive resilient holistic supply chain networks. Int. J. Adv. Manuf. Technol. 69, 427–437 (2013)

    Article  Google Scholar 

  46. Raj, R., Wang, J.W., Nayak, A., Tiwari, M.K., Han, B., Liu, C.L., Zhang, W.J.: Measuring the resilience of supply chain systems using a survival model. IEEE Syst. J. 9(2), 377–381 (2014)

    Article  ADS  Google Scholar 

  47. Guo, P., Hou, W., Guo, L., Sun, W., Liu, C., Bao, H., Duong, L.H.K., Liu, W.: Fault-tolerant routing mechanism in 3d optical network-on-chip based on node reuse. IEEE Trans. Parallel Distrib. Syst. 31(3), 547–564 (2019)

    Article  Google Scholar 

  48. Kakadia, D., Yang, J., Gilgur, A.: Evolved universal terrestrial radio access network (EUTRAN). In: Network Performance and Fault Analytics for LTE Wireless Service Providers, pp. 61–81. Springer, New Delhi (2017). https://doi.org/10.1007/978-81-322-3721-1_3

  49. You, C., Huang, K., Chae, H., Kim, B.-H.: Energy-efficient resource allocation for mobile-edge computation offloading. IEEE Trans. Wirel. Commun. 16(3), 1397–1411 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Department of Supply Chain and Information Management & Big Data Intelligence Centre, The Hang Seng University of Hong Kong, and the Department of Industrial and Systems Engineering & Research Institute for Advanced Manufacturing, The Hong Kong Polytechnic University for supporting this research study.

Funding

The work described in this paper was partly supported by a grant from Research Institute for Advanced Manufacturing, The Hong Kong Polytechnic University (Project code: CD4E) and a grant from the University Grants Committee of the HK SAR, China (RMGS Project Acc. No.: 700043).

Author information

Authors and Affiliations

  1. Department of Industrial and Systems Engineering, Research Institute for Advanced Manufacturing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong

    Y. P. Tsang, C. K. M. Lee & Kening Zhang

  2. Department of Supply Chain and Information Management, The Hang Seng University of Hong Kong, Shatin, Hong Kong

    C. H. Wu & W. H. Ip

  3. Department of Mech. Engg., The University of Saskatchewan, Saskatoon, Canada

    W. H. Ip

Authors
  1. Y. P. Tsang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. K. M. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kening Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. C. H. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. W. H. Ip
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualisation, Y.P.T. and C.K.M.L.; Methodology, Y.P.T. and K.Z.; Formal analysis, Y.P.T. and C.H.W.; Data curation, K.Z. and C.H.W.; Writing—original draft preparation, Y.P.T. and K.Z.; Writing—review and editing, C.H.W., W.H.I., C.K.M.L.; Project administration, C.K.M.L. and W.H.I.; funding acquisition, C.K.M.L. and C.H.W.; All authors reviewed the manuscript before re-submission.

Corresponding author

Correspondence to C. H. Wu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsang, Y.P., Lee, C.K.M., Zhang, K. et al. On-Chain and Off-Chain Data Management for Blockchain-Internet of Things: A Multi-Agent Deep Reinforcement Learning Approach. J Grid Computing 22, 16 (2024). https://doi.org/10.1007/s10723-023-09739-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-023-09739-x

Keywords

Search

Navigation