Skip to main content
SpringerLink
Log in

A Method of Data Integrity Check and Repair in Big Data Storage Platform

  • Conference paper
  • First Online:
Bio-inspired Information and Communication Technologies (BICT 2020)

Abstract

In the big data storage platform, in order to ensure the security of user data, it is necessary to perform cyclic verification on the stored data and repair the damaged data in time. Considering the problems of low verification efficiency, low check frequency and low calibration accuracy of HDFS data integrity check, this paper proposed a new HDFS storage platform security check and repair scheme. The process can effectively reduce the amount of calculation and communication overhead, and can support the dynamic operation of the data. Experiments show that this method has certain advantages in security, scalability and flexibility.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wang, Y., et al.: Data integrity checking with reliable data transfer for secure cloud storage. Int. J. Web Grid Serv. 14(1), 106–121 (2018)

    Article  Google Scholar 

  2. Pardeshi, P.M., Tidke, B.: Improvement of data integrity and data dynamics for data storage security in cloud computing. In: Mandal, J.K., Satapathy, S.C., Sanyal, M.K., Sarkar, P.P., Mukhopadhyay, A. (eds.) Information Systems Design and Intelligent Applications. AISC, vol. 339, pp. 279–289. Springer, New Delhi (2015). https://doi.org/10.1007/978-81-322-2250-7_27

    Chapter  Google Scholar 

  3. Liu, C., Yang, C., Zhang, X., et al.: External integrity verification for outsourced big data in cloud and IoT: a big picture. Future Gen. Comput. Syst. 49, 58–67 (2015)

    Article  Google Scholar 

  4. Maneas, S., Schroeder, B.: The evolution of the hadoop distributed file system. In: 2018 32nd International Conference on Advanced Information Networking and Applications Workshops (WAINA). IEEE Computer Society (2018)

    Google Scholar 

  5. Dinsmore, T.W.: The Hadoop Ecosystem. Disruptive Analytics. Apress (2016)

    Google Scholar 

  6. Muthuram, R., Kousalya, G.: A survey on integrity verification and data auditing schemes for data verification in remote cloud servers. Electron. Gov. Int. J. 13(1), 408–418 (2017)

    Google Scholar 

Download references

Acknowledgment

This research was funded by the National Key Research and Development Program of China under Grant 2016QY06X1203 and the National Natural Science Foundation of China under Grant 61701019.

Author information

Authors and Affiliations

  1. School of Electronics and Information Engineering, Beijing Jiaotong University, Beijing, 100044, China

    Jiaxin Li, Yun Liu & Zhenjiang Zhang

  2. Department of Electrical Engineering, National Dong Hwa University, Hualien, 974, Taiwan

    Han-Chieh Chao

Authors
  1. Jiaxin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenjiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Han-Chieh Chao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiaxin Li .

Editor information

Editors and Affiliations

  1. University of Electronic Science and Technology, Chengdu, China

    Yifan Chen

  2. Osaka University, Osaka, Japan

    Tadashi Nakano

  3. Tongji University, Shanghai, China

    Lin Lin

  4. Resch School of Engineering, University of Wisconsiin-Green Bay, Green Bay, WI, USA

    Mohammad Upal Mahfuz

  5. School of Engineering, Cranfield University, Cranfield, UK

    Weisi Guo

Rights and permissions

Reprints and permissions

Copyright information

© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, J., Liu, Y., Zhang, Z., Chao, HC. (2020). A Method of Data Integrity Check and Repair in Big Data Storage Platform. In: Chen, Y., Nakano, T., Lin, L., Mahfuz, M., Guo, W. (eds) Bio-inspired Information and Communication Technologies. BICT 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 329. Springer, Cham. https://doi.org/10.1007/978-3-030-57115-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-57115-3_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-57114-6

  • Online ISBN: 978-3-030-57115-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation