Skip to main content

Advertisement

Springer Nature Link
Log in

Dynamic data auditing scheme for big data storage

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

When users store data in big data platforms, the integrity of outsourced data is a major concern for data owners due to the lack of direct control over the data. However, the existing remote data auditing schemes for big data platforms are only applicable to static data. In order to verify the integrity of dynamic data in a Hadoop big data platform, we presents a dynamic auditing scheme meeting the special requirement of Hadoop. Concretely, a new data structure, namely Data Block Index Table, is designed to support dynamic data operations on HDFS (Hadoop distributed file system), including appending, inserting, deleting, and modifying. Then combined with the MapReduce framework, a dynamic auditing algorithm is designed to audit the data on HDFS concurrently. Analysis shows that the proposed scheme is secure enough to resist forge attack, replace attack and replay attack on big data platform. It is also efficient in both computation and communication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chaudhary R, Aujla G S, Garg S, Kumar N, Rodrigues J. SDN-enabled multi-attribute-based secure communication for smart grid in IIoT environment. IEEE Transactions on Industrial Informatics, 2018, 14(6): 2629–2640

    Article  Google Scholar 

  2. Chaudhary R, Aujla G S, Kumar N, Rodrigues J. Optimized big data management across multi-cloud data centers: software-definednetwork-based analysis. IEEE Communications Magazine, 2018, 56(2): 118–126

    Article  Google Scholar 

  3. Ateniese G, Burns R, Curtmola R, Herring J, Kissner L, Peterson Z, Song D. Provable data possession at untrusted stores. In: Proceedings of the 14th ACM Conference on Computer and Communications Security. 2007, 598–609

    Google Scholar 

  4. Ateniese G, Pietro R D, Mancini L V, Tsudik. Scalable and efficient provable data possession. In: Proceedings of the 4th International Conference on Security and Privacy in Communication Networks Conference. 2008

    Google Scholar 

  5. Juels A, Kaliski B S. PORS: proofs of retrievability for large files. In: Proceedings of the 14th ACM Conference on Computer and Communication Security Conference. 2007, 584–597

    Google Scholar 

  6. Shacham H, Waters B. Compact proofs of retrievability. In: Proceedings of the 14th International Conference on the Theory and Application of Cryptology and Information Security. 2008, 90–107

    MATH  Google Scholar 

  7. Erway C, Kupcu A, Papamanthou C, Tamassia R. Dynamic provable data possession. In: Proceedings of the 16th ACM Conference on Computer Communication Security. 2009, 213–222

    Google Scholar 

  8. Wang Q, Wang C, Li J, Ren K, Lou W. Enabling public verifiability and data dynamics for storage security in cloud computing. In: Proceedings of European Symposium on Research in Computer Security. 2009, 355–370

    Google Scholar 

  9. Wang C, Wang Q, Ren K, Lou W. Privacy-preserving public auditing for data storage security in cloud computing. In: Proceedings of the 29th Conference on Information Communications. 2010, 1–9

    Google Scholar 

  10. Wang Q, Wang C, Ren K, Lou W, Li J. Enabling public auditability and data dynamics for storage security in cloud computing. IEEE Transactions on Parallel and Distributed Systems, 2011, 22(5): 847–859

    Article  Google Scholar 

  11. Zhu Y, Hu H, Ahn G J, Yu M. Cooperative provable data possession for integrity verification in multi-cloud storage. IEEE Transactions on Parallel and Distributed Systems, 2012, 23(12): 2231–2243

    Article  Google Scholar 

  12. Zhu Y, Ahn G J, Hu H, Yau S S, An H G, Hu C J. Dynamic audit services for outsourced storages in clouds. IEEE Transactions on Services Computing, 2012, 6(2): 227–238

    Google Scholar 

  13. Li J, Xie D, Cai Z. Secure auditing and deduplicating data in cloud. IEEE Transactions on Computers, 2016, 65(8): 2386–2396

    Article  MathSciNet  Google Scholar 

  14. Sookhak M, Akhunzada A, Gani A, Khan M K, Anuar N B. Towards dynamic remote data auditing in computational clouds. The Scientific World Journal, 2014, 2014: 269357

    Article  Google Scholar 

  15. Yang K, Jia X. An efficient and secure dynamic auditing protocol for data storage in cloud computing. IEEE Transactions on Parallel and Distributed Systems, 2013, 24(9): 1717–1726

    Article  Google Scholar 

  16. Aujla G S, Chaudhary R, Kumar N, Das A K, Rodrigues J. SecSVA: secure storage, verification, and auditing of big data in the cloud environment. IEEE Communications Magazine, 2018, 56(1): 78–85

    Article  Google Scholar 

  17. Garg N, Bawa S. RITS-MHT: relative indexed and time stamped Merkle hash tree based data auditing protocol for cloud computing. Journal of Network and Computer Applications, 2017, 84: 1–13

    Article  Google Scholar 

  18. Chen X, Li J, Weng J, Ma J, Lou W. Verifiable computation over large database with incremental updates. IEEE Transactions on Computers, 2016, 65(10): 3184–3195

    Article  MathSciNet  Google Scholar 

  19. Chen X, Li J, Huang X, Ma J, Lou W. New publicly verifiable databases with efficient updates. IEEE Transactions on Dependable and Secure Computing, 2015, 12(5): 546–556

    Article  Google Scholar 

Download references

Acknowledgements

Project supported by the National Key Research and Development Program of China (2016YFC1000307) for valuable helps.

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Beihang University, Beijing, 100083, China

    Xingyue Chen

  2. School of Cyber Science and Technology, Beihang University, Beijing, 100083, China

    Tao Shang, Feng Zhang, Jianwei Liu & Zhenyu Guan

Authors
  1. Xingyue Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Tao Shang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Feng Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jianwei Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Zhenyu Guan
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Zhenyu Guan.

Additional information

Xingyue Chen received the BS degree in Electronic and Information Engineering from Beihang University, China in 2016. She is currently pursuing the MS degree in the School of Electronic and Information Engineering at Beihang University. Her research interests include big data security and privacy protection.

Tao Shang received his PhD degree in System Engineering from Kochi University of Technology, Japan in 2006. From September 2007 to September 2009, he worked as a postdoctoral in the School of Computer Science at Beihang University, China. Now he is an associate professor of School of Cyber Science and Technology at Beihang University, China. His current research interests include network security and quantum cryptography.

Feng Zhang will receive the BS degree in Electronic and Information Engineering from Beihang University, China in 2018. He is currently admitted to pursue the MS degree in the School of Cyber Science and Technology at Beihang University. His research interests big data security.

Jianwei Liu received his PhD degree in Communication and Electronic System from Xidian University, China in 1998. Now he is a professor of School of Cyber Science and Technology at Beihang University, China. His current research interests include wireless communication network, coding theory, and information security.

Zhenyu Guan received his PhD degree in Electronic Engineering from Imperial College London, UK in 2013. From 2013, he became a lecturer of School of Cyber Science and Technology at Beihang University, China. His current research interests include security engineering and cryptography.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, X., Shang, T., Zhang, F. et al. Dynamic data auditing scheme for big data storage. Front. Comput. Sci. 14, 219–229 (2020). https://doi.org/10.1007/s11704-018-8117-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-018-8117-6

Keywords