Skip to main content
SpringerLink
Log in

A generic approach for integrity verification of big data

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Data integrity verification is a primary issue for the prominent and unchallenged success of Big Data. The large scale of data includes extra security overheads on customers that used Big Data services. In Big Data environment, many security challenges have pinched out such as data security, malicious insider attack, cyber attack, and abuse of data. In this paper, we have analyzed and identified a novel data integrity verification technique that provides better performance, efficiency, and quick review of Big Data. The milestone of our work are algebraic signature, homomorphic tag, and Combinatorial Batch Codes (CBC). Homomorphic tag delegates a special verifiable value to each data blocks. CBC allocates integral data to store on Big Data servers. This research work used multiple third-party auditors for batch auditing which ensure data integrity in a Big Data environment. Our approach handles single point failure error of single Third Party Auditor (TPA) efficiently. Without using any additional data structure, it supports dynamic data operations. The security and performance analysis with comparative results show the genuine application of our approach within the latest Big Data environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Ruj, S., Saxena, R.: Securing cloud data. In: Terzo, O., Mossucca, L. (eds.) Cloud Computing with e-science Applications, pp. 41–72. CRC Press, Boca Raton (2015)

    Google Scholar 

  2. Saxena, R., Dey, S.: A curious collaborative approach for data integrity verification in cloud computing. CSI Trans. ICT 5, 407–418 (2017)

    Article  Google Scholar 

  3. Saxena, R., Dey, S.: Collaborative approach for data integrity verification in cloud computing. In: Proceedings of the SNDS, Communications in Computer and Information Science, vol. 420, pp. 1–15. Springer (2014)

  4. Saxena, R., Dey, S.: Cloud audit: a data integrity verification approach for cloud computing. Proc. Comput. Sci. 89, 142–151 (2016)

    Article  Google Scholar 

  5. Shacham, H., Waters, B.: Compact proofs of retrievability. IACR Cryptol. ePrint Arch. 2008, 73 (2008)

    MATH  Google Scholar 

  6. Wang, C., Chow, S.S., Wang, Q., Ren, K., Lou, W.: Privacy-preserving public auditing for secure cloud storage. IEEE Trans. Comput. 23(12), 2231–2244 (2014)

    MATH  Google Scholar 

  7. Zhu, Y., Hu, H., Ahn, G.J., Yu, M.: Cooperative provable data possession for integrity verification in multicloud storage. IEEE Trans. Parallel Distrib. Syst. 23(12), 2231–2244 (2012)

    Article  Google Scholar 

  8. Erway, C.C., Küpçü, A., Papamanthou, C., Tamassia, R.: Dynamic provable data possession. In: Proceedings of the ACM Conference on Computer and Communications Security, pp. 213–222 (2009)

  9. Juels, A., Bowers, K.D., Oprea, A.: Hail: a high-availability and integrity layer for cloud storage. In: Proceedings of the ACM Conference on Computer and Communications Security. pp. 187–198 (2009)

  10. Dodis, Y., Vadhan, S.P., Wichs, D.: Proofs of retrievability via hardness amplification. In: Proceedings of the TCC. pp. 109–127 (2009)

  11. Wang, Q., Wang, C., Li, J., Ren, K., Lou, W.: Enabling public verifiability and data dynamics for storage security in cloud computing. In: Proceedings of the Computer Security–ESORICS 2009, pp. 355–370. Springer (2009)

  12. Bowers, K.D., Juels, A., Oprea, A.: Proofs of retrievability: Theory and implementation. In: Proceedings of the 2009 ACM workshop on Cloud computing security, pp. 43–54. ACM (2009)

  13. Litwin, W., Schwarz, T.: Algebraic signatures for scalable distributed data structures. In: Proceedings of the 20th International Conference on Data Engineering, 2004, pp. 412–423. IEEE (2004)

  14. Schwarz, T.S., Miller, E.L.: Store, forget, and check: Using algebraic signatures to check remotely administered storage. In: Proceedings of the 26th IEEE International Conference on Distributed Computing Systems, 2006, pp. 123–129. IEEE (2006)

  15. Stinson, D., Wei, R., Paterson, M.B.: Combinatorial batch codes. Adv. Math. Commun. 3(1), 13–27 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  16. Bujtás, C., Tuza, Z.: Optimal combinatorial batch codes derived from dual systems. Miskolc Math. Notes 12(1), 11–23 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  17. Merkle, R.: Protocols for public key cryptosystems. In: Proceedings of the IEEE Symposium on Security and Privacy, pp. 2231–2244 (1980)

  18. Brualdi, R.A., Kiernan, K.P., Meyer, S.A., Schroeder, M.W.: Combinatorial batch codes and transversal matroids. Adv. Math. Commun 4(3), 419–431 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  19. AWS. http://xenserver.org/open-source-virtualization-download.html (2014). Accessed 08 October 2014

  20. Xenserver. http://xenserver.org/discuss-virtualization/virtualization-blog.html (2014). Accessed 08 October 2014

  21. Cloudera. http://www.cloudera.com/content/cloudera/en/downloads.html (2014). Accessed 08 October 2014

  22. Chen, L.: Using algebraic signatures to check data possession in cloud storage. Fut. Gener. Comput. Syst. 29(7), 1709–1715 (2013)

    Article  Google Scholar 

  23. Etemad, M., Küpçü, A.: Transparent, Distributed, and Replicated Dynamic Provable Data Possession, pp. 1–18. Springer, Berlin (2013)

    Google Scholar 

  24. Wang, C., Wang, Q., Ren, K., Lou, W.: Privacy-preserving public auditing for data storage security in cloud computing. In: INFOCOM, pp. 525–533 (2010)

  25. Hao, Z., Zhong, S., Yu, N.: A privacy-preserving remote data integrity checking protocol with data dynamics and public verifiability. IEEE Trans. Knowl. Data Eng. 23(9), 1432–1437 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Indian Institute of Technology Indore for their financial and infrastructure support.

Author information

Authors and Affiliations

  1. Big Data Research Lab, Department of Computer Science and Engineering, Indian Institute of Technology Indore, Indore, India

    Rajat Saxena & Somnath Dey

Authors
  1. Rajat Saxena
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Somnath Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rajat Saxena.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saxena, R., Dey, S. A generic approach for integrity verification of big data. Cluster Comput 22, 529–540 (2019). https://doi.org/10.1007/s10586-018-2861-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-018-2861-0

Keywords

Search

Navigation