research-article

FASTSync: A FAST Delta Sync Scheme for Encrypted Cloud Storage in High-bandwidth Network Environments

Authors:

Bo MaoAuthors Info & Claims

ACM Transactions on Storage, Volume 19, Issue 4

Article No.: 37, Pages 1 - 22

https://doi.org/10.1145/3607536

Published: 03 October 2023 Publication History

Abstract

More and more data are stored in cloud storage, which brings two major challenges. First, the modified files in the cloud should be quickly synchronized to ensure data consistency, e.g., delta synchronization (sync) achieves efficient cloud sync by synchronizing only the updated part of the file. Second, the huge data in the cloud needs to be deduplicated and encrypted, e.g., Message-Locked Encryption (MLE) implements data deduplication by encrypting the content among different users. However, when combined, a few updates in the content can cause large sync traffic amplification for both keys and ciphertext in the MLE-based cloud storage, significantly degrading the cloud sync efficiency. A feature-based encryption sync scheme, FeatureSync, is proposed to address the delta amplification problem. However, with further improvement of the network bandwidth, the performance of FeatureSync stagnates. In our preliminary experimental evaluations, we find that the bottleneck of the computational overhead in the high-bandwidth network environments is the main bottleneck in FeatureSync. In this article, we propose an enhanced feature-based encryption sync scheme FASTSync to optimize the performance of FeatureSync in high-bandwidth network environments. The performance evaluations on a lightweight prototype implementation of FASTSync show that FASTSync reduces the cloud sync time by 70.3% and the encryption time by 37.3%, on average, compared with FeatureSync.

References

[1]

Apple iCloud. 2022. Retrieved from https://www.icloud.com/.

[2]

Mihir Bellare, Sriram Keelveedhi, and Thomas Ristenpart. 2013. Message-locked encryption and secure deduplication. In Proceedings of the 32nd Annual International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT’13).

[3]

Deepak R. Bobbarjung, Suresh Jagannathan, and Cezary Dubnicki. 2006. Improving duplicate elimination in storage systems. ACM Trans. Storage 2, 4 (2006), 424–448.

Digital Library

[4]

Boxcrypto. 2022. Retrieved from https://www.boxcryptor.com/en/.

[5]

Andrei Z. Broder. 1997. On the resemblance and containment of documents. In Proceedings of the Compression and Complexity of SEQUENCES.

Digital Library

[6]

Linux Traffic Control. 2022. Retrieved from https://tldp.org/HOWTO/Traffic-Control-HOWTO/intro.html.

[7]

Yong Cui, Zeqi Lai, Xin Wang, Ningwei Dai, and Congcong Miao. 2015. QuickSync: Improving synchronization efficiency for mobile cloud storage services. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (MobiCom’15).

Digital Library

[8]

John R. Douceur, Atul Adya, William J. Bolosky, P. Simon, and Marvin Theimer. 2002. Reclaiming space from duplicate files in a serverless distributed file system. In Proceedings of the 22nd International Conference on Distributed Computing Systems (ICDCS’02).

[9]

Fred Douglis. 2003. Application-specific delta-encoding via resemblance detection. In Proceedings of the USENIX Annual Technical Conference (ATC’03).

[10]

Dropbox. 2022. Retrieved from: http://www.dropboxchina.com/.

[11]

Sookasa’s encrypted cloud storage. 2015. Retrieved from: https://www.sookasa.com/resources/encrypted-cloud-storage/.

[12]

FastCDC. 2022. Retrieved from: https://github.com/Quicksandlyh/FastCDC-c/.

[13]

Seafile: Enterprise file sync, share platform with high reliability, and performance. 2022. Retrieved from: https://www.seafile.com/en/home.

[14]

Github. 2022. Retrieved from: https://github.com/.

[15]

Google Drive. 2022. Retrieved from: https://drive.google.com/.

[16]

HashMap. 2022. Retrieved from: https://github.com/tidwall/hashmap.c/.

[17]

Yuan He, Lingfeng Xiang, Wen Xia, Hong Jiang, Zhenhua Li, Xuan Wang, and Xiangyu Zou. 2020. Dsync: A lightweight delta synchronization approach for cloud storage services. In Proceedings of the 36th Symposium on Mass Storage Systems and Technologies (MSST’20).

[18]

Sriram Keelveedhi, Mihir Bellare, and Thomas Ristenpart. 2013. DupLESS: Server-aided encryption for deduplicated storage. In Proceedings of the 22th USENIX Security Symposium (Security’13).

[19]

Linux Kernel. 2022. Retrieved from: https://www.kernel.org/.

[20]

Purushottam Kulkarni, Fred Douglis, Jason D. Lavoie, and John M. Tracey. 2004. Redundancy elimination within large collections of files. In Proceedings of the USENIX Annual Technical Conference (ATC’04).

Digital Library

[21]

Jingwei Li, Patrick P. C. Lee, Yanjing Ren, and Xiaosong Zhang. 2019. Metadedup: Deduplicating metadata in encrypted deduplication via indirection. In Proceedings of the 35th Symposium on Mass Storage Systems and Technologies (MSST’19).

[22]

Jingwei Li, Chuan Qin, Patrick P. C. Lee, and Xiaosong Zhang. 2017. Information leakage in encrypted deduplication via frequency analysis. In Proceedings of the 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’17).

[23]

Jingwei Li, Zuoru Yang, Yanjing Ren, Patrick P. C. Lee, and Xiaosong Zhang. 2020. Balancing storage efficiency and data confidentiality with tunable encrypted deduplication. In Proceedings of the 15th EuroSys Conference (EuroSys’20).

Digital Library

[24]

Zhenhua Li, Christo Wilson, Zhefu Jiang, Yao Liu, Ben Y. Zhao, Cheng Jin, Zhi-Li Zhang, and Yafei Dai. 2013. Efficient batched synchronization in Dropbox-like cloud storage services. In Proceedings of the 14th International Middleware Conference (Middleware’13).

[25]

J. MacDonald. 2000. File System Support for Delta Compression. Master’s thesis. Department of Electrical Engineering and Computer Science, University of California at Berkeley.

[26]

Bo Mao, Hong Jiang, Suzhen Wu, and Lei Tian. 2016. Leveraging data deduplication to improve the performance of primary storage systems in the cloud. IEEE Trans. Comput. 65, 6 (2016), 1775–1788.

Digital Library

[27]

Bo Mao, Suzhen Wu, and Hong Jiang. 2016. Exploiting workload characteristics and service diversity to improve the availability of cloud storage systems. IEEE Trans. Parallel Distrib. Syst. 27, 7 (2016), 2010–2021.

Digital Library

[28]

Microsoft OneDrive. 2022. Retrieved from: https://drive.google.com/.

[29]

Irfan Mohiuddin, Ahmad Almogren, Mohammed Al Qurishi, Mohammad Mehedi Hassan, Iehab Al Rassan, and Giancarlo Fortino. 2019. Secure distributed adaptive bin packing algorithm for cloud storage. Fut. Gen. Comput. Syst. 90 (2019), 307–316.

[30]

Athicha Muthitacharoen, Benjie Chen, and David Mazières. 2001. A low-bandwidth network file system. In Proceedings of the 18th ACM Symposium on Operating System Principles (SOSP’01).

Digital Library

[31]

OpenSSL. 2022. Retrieved from: https://github.com/openssl/openssl.

[32]

Chuan Qin, Jingwei Li, and Patrick P. C. Lee. 2019. The design and implementation of a rekeying-aware encrypted deduplication storage system. ACM Trans. Storage 13, 1 (2017), 1–30.

Digital Library

[33]

Philip Shilane, Mark Huang, Grant Wallace, and Windsor Hsu. 2012. WAN-optimized replication of backup datasets using stream-informed delta compression. ACM Trans. Storage 8, 4 (2012), 1–26.

Digital Library

[34]

Torsten Suel and Nasir Memon. 2002. Algorithms for delta compression and remote file synchronization. In Handbook of Lossless Compression, S. Sayood (Ed.). Academic Press.

[35]

Suzhen Wu, Longquan Liu, Hong Jiang, Hao Che, and Bo Mao. 2019. PandaSync: Network and workload aware hybrid cloud sync optimization. In Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS’19).

[36]

Suzhen Wu, Bo Mao, Hong Jiang, Huagao Luan, and Jindong Zhou. 2019. PFP: Improving the reliability of deduplication-based storage systems with per-file parity. IEEE Trans. Parallel Distrib. Syst. 30, 9 (2019), 2117–2129.

Digital Library

[37]

Suzhen Wu, Zhanhong Tu, Zuocheng Wang, Zhirong Shen, and Bo Mao. 2021. When delta sync meets message-locked encryption: A feature-based delta sync scheme for encrypted cloud storage. In Proceedings of the 41st IEEE International Conference on Distributed Computing Systems (ICDCS’21). 337–347.

[38]

Wen Xia, Hong Jiang, Dan Feng, F. Douglis, P. Shilane, Yu Hua, Min Fu, Yucheng Zhang, and Yukun Zhou. 2016. A comprehensive study of the past, present, and future of data deduplication. Proc. IEEE 104 (2016), 1681–1710.

[39]

Wen Xia, Hong Jiang, Dan Feng, Lei Tian, Min Fu, and Yukun Zhou. 2014. Ddelta: A deduplication-inspired fast delta compression approach. Perform. Eval. 79 (2014), 258–272.

[40]

Wen Xia, Yukun Zhou, Hong Jiang, Dan Feng, Yu Hua, Yuchong Hu, Qing Liu, and Yucheng Zhang. 2016. FastCDC: A fast and efficient content-defined chunking approach for data deduplication. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC’16), Ajay Gulati and Hakim Weatherspoon (Eds.). 101–114.

[41]

He Xiao, Zhenhua Li, Ennan Zhai, Tianyin Xu, Yang Li, Yunhao Liu, Quanlu Zhang, and Yao Liu. 2018. Towards web-based delta synchronization for cloud storage services. In Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST’18).

[42]

Quanlu Zhang, Zhenhua Li, Zhi Yang, Shenglong Li, and Yafei Dai. 2017. DeltaCFS: Boosting delta sync for cloud storage services by learning from NFS. In Proceedings of the 37th IEEE International Conference on Distributed Computing Systems (ICDCS’17).

[43]

Yukun Zhou, Dan Feng, Wen Xia, Min Fu, and Chunguang Li. 2015. SecDep: A user-aware efficient fine-grained secure deduplication scheme with multi-level key management. In Proceedings of the IEEE 31st Symposium on Mass Storage Systems and Technologies (MSST’15).

Cited By

Zhang MLu LWu YYan ZSun JLin FRen K(2025)DroneAudioID: A Lightweight Acoustic Fingerprint-Based Drone Authentication System for Secure Drone DeliveryIEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.352781420(1447-1461)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1109/TIFS.2025.3527814
Sun TJiang BLi BLv JGao YDong WBagchi SZhang Y(2024)SimEncProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692030(615-630)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691992.3692030
Wang YLi YZhang XLiu XDai AChan ACui Z(2024)Edit Temporal-Consistent Videos with Image Diffusion ModelACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369134420:12(1-16)Online publication date: 26-Nov-2024
https://dl.acm.org/doi/10.1145/3691344
Show More Cited By

Index Terms

FASTSync: A FAST Delta Sync Scheme for Encrypted Cloud Storage in High-bandwidth Network Environments
1. Information systems
  1. Data management systems
    1. Information integration
      1. Deduplication
  2. Information storage systems
    1. Storage architectures
      1. Cloud based storage

Recommendations

RAID-II: A Scalable Storage Architecture for High-Bandwidth Network
Encrypted Data Deduplication in Cloud Storage
ASIAJCIS '15: Proceedings of the 2015 10th Asia Joint Conference on Information Security

Cloud storage is a remote storage service, where users can upload and download their data anytime and anywhere. However, it raises issues regarding privacy and data confidentiality because all the data are stored in the cloud storage. This is a subject ...
Efficient update of encrypted files for cloud storage
UCC '15: Proceedings of the 8th International Conference on Utility and Cloud Computing

We propose a method for file encryption based on AES-CTR suitable for cloud storage. Our method allows efficient updates of encrypted files by minimizing the amount of data that need to be re-encrypted. It achieves significantly better performance than ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage

ACM Transactions on Storage Volume 19, Issue 4

November 2023

238 pages

ISSN:1553-3077

EISSN:1553-3093

DOI:10.1145/3626486

Editor:
Erez Zadok
Stony Brook University, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 October 2023

Online AM: 07 July 2023

Accepted: 20 June 2023

Revised: 22 March 2023

Received: 27 September 2022

Published in TOS Volume 19, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Open Project Program of Wuhan National Laboratory for Optoelectronics
Research Project of Zhejiang Lab

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
399
Total Downloads

Downloads (Last 12 months)176
Downloads (Last 6 weeks)11

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang MLu LWu YYan ZSun JLin FRen K(2025)DroneAudioID: A Lightweight Acoustic Fingerprint-Based Drone Authentication System for Secure Drone DeliveryIEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.352781420(1447-1461)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1109/TIFS.2025.3527814
Sun TJiang BLi BLv JGao YDong WBagchi SZhang Y(2024)SimEncProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692030(615-630)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691992.3692030
Wang YLi YZhang XLiu XDai AChan ACui Z(2024)Edit Temporal-Consistent Videos with Image Diffusion ModelACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369134420:12(1-16)Online publication date: 26-Nov-2024
https://dl.acm.org/doi/10.1145/3691344
Li JTan YYang JLi ZYe HXia CLi Y(2024)Unsupervised Adversarial Example Detection of Vision Transformers for Trustworthy Edge ComputingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3674981Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1145/3674981
Wu HWang ZLi YLiu XLee T(2024)Suitable and Style-Consistent Multi-Texture Recommendation for Cartoon IllustrationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251820:7(1-26)Online publication date: 16-May-2024
https://dl.acm.org/doi/10.1145/3652518
Suo YZheng ZWang XZhang BYang Y(2024)Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video GenerationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364836820:6(1-18)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3648368
Shi PHu MShi XRen F(2024)Deep Modular Co-Attention Shifting Network for Multimodal Sentiment AnalysisACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363470620:4(1-23)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3634706
Ma ZWang SLuo XGu ZChen CLi JHua XLu G(2024)HARR: Learning Discriminative and High-Quality Hash Codes for Image RetrievalACM Transactions on Multimedia Computing, Communications, and Applications10.1145/362716220:5(1-23)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3627162
Wu XFeng X(2024)Size Invariant Visual Cryptography Schemes With Evolving Threshold Access StructuresIEEE Transactions on Multimedia10.1109/TMM.2023.328257326(1488-1503)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3282573
Su QWu DWu CLi BWang W(2024)From Data to Optimization: Data-Free Deep Incremental Hashing With Data Disambiguation and Adaptive ProxiesIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.336797434:7(6576-6589)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3367974
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents