Skip to main content
SpringerLink
Log in

AIR: an approximate intelligent redistribution approach to accelerate RAID scaling

  • Regular Paper
  • Published:
CCF Transactions on High Performance Computing Aims and scope Submit manuscript

Abstract

Nowadays, videos and images are becoming the predominant format of data storage, which take up more space than conventional plain texts. This rapid increment brings a high requirement on scalability in large data centers, where disk arrays (also referred to as “RAID”) are the main devices to store the numerous data. To improve the scalability of RAID systems, several scaling approaches are proposed to guarantee a uniform data distribution, such as decreasing migration I/Os and speeding up the scaling process. However, typical approaches are offline and ignore the impacts of concurrent application I/Os, which plays an important role on RAID scaling (i.e., data migration, data distribution, etc.). For example, an exact uniform data distribution among disks doesn’t mean an even I/O accesses to these disks. To address this problem, in this paper, we propose an approximate intelligent redistribution (AIR) approach to accelerate RAID scaling. The main idea of AIR is utilizing the dynamic data access patterns from concurrent application workloads, and providing an approximate data distribution to guarantee a uniform I/O accesses to various data disks. To achieve this goal, AIR utilizes the prevailing machine learning algorithms to identify hot data from application workloads, and gives an intelligent migration approach to minimize the data movements. By this way, AIR can sharply cut down the migration I/Os. To demonstrate the effectiveness of AIR approach, we conduct several simulations via Disksim. The results show that, compared to traditional RAID scaling approaches such as FastScale and GSR, AIR saves up to 99.3% I/O cost and reduces the data migration ratio by up to 95.8%, which speeds up the scaling process by a factor of up to 30.3X.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  • Brown, N.: Online RAID-5 resizing. drivers/md/raid5.c in the source code of Linux Kernel 2.6.18. http://www.kernel.org/ (2006). Accessed 6 May 2019

  • Bucy, J., Schindler, J., Schlosser, S., Ganger, G.: The disksim simulation environment version 4.0 reference manual (cmu-pdl-08-101). Parallel Data Laboratory (2008)

  • Chen, P., Lee, E., et al.: RAID: high-performance, reliable secondary storage. ACM Comput. Surv. 26(2), 145–185 (1994)

    Article  Google Scholar 

  • Corbett, P., et al.: Row-diagonal parity for double disk failure correction. In: Proc. of the FAST ’04 (2004)

  • Goel, A., et al.: SCADDAR: An efficient randomized technique to reorganize continuous media blocks. In: Proc. of the ICDE’02 (2002)

  • Gonzalez, J., Corte, T.: Increasing the capacity of RAID-5 by online gradual assimilation. In: Proc. of the SNAPI’04 (2004)

  • Gonzalez, J., Cortes, T.: Increasing the capacity of RAID-5 by online gradual assimilation. In: Proc. of the SNAPI ’04 (2004)

  • Gu, J., Wu, C., Xie, X., Qiu, H., Li, J., Guo, M., He, X., Dong, Y., Zhao, Y.: Optimizing the parity-check matrix for efficient decoding of RS-based cloud storage systems. In: The 32nd IEEE International Parallel and Distributed Processing Symposium (IPDPS 2019). IEEE (2019)

  • Hetzler, S.: Storage array scaling method and system with minimal data movement. US Patent 20080276057, June (2008)

  • Hitachi Data Systems, Hitachi Virtual Storage Platform Architecture Guide. (2011) http://www.hds.com/assets/pdf/hitachi-architecture-guide-virtual-storage-platform.pdf?WT.ac=uk_hp_sp2r1. Accessed 6 May 2019

  • Huang, C., Xu, L.: STAR: an efficient coding scheme for correcting triple storage node failures. IEEE Trans. Comput. 57(7), 889–901 (2008)

    Article  MathSciNet  Google Scholar 

  • Hwang, K., et al.: RAID-x: A new distributed disk array for I/O-centric cluster computing. In: Proc. of the HPDC ’00 (2000)

  • Jiang, Y., Wu, C., Li, J., et al.: BDR: A balanced data redistribution scheme to accelerate the scaling process of XOR-based triple disk failure tolerant arrays. In: 2016 IEEE 34th International Conference on Computer Design (ICCD). IEEE Computer Society (2016)

  • Jin, H., Wu, C., Xie, X., Li, J., Guo, M., Lin, H., Zhang, J.: Approximate code: a cost-effective erasure coding framework for tiered video storage in cloud systems. In: 48th International Conference on Parallel Processing (ICPP 2019), August 5–8, 2019, Kyoto, Japan. ACM, New York, NY, USA, 10 p. https://doi.org/10.1145/3337821.3337869 (2019)

  • Miranda, A., Cortes, T.: Craid: online raid upgrades using dynamic hot data reorganization. In: Proc. of the USENIX FAST ’14 (2014)

  • Oracle Corporation. A better RAID strategy for high capacity drives in mainframe storage. http://www.oracle.com/technetwork/articles/systems-hardware-architecture/raid-strategy-hi-capacity-drives-170907.pdf (2013). Accessed 3 May 2019

  • Pareto, V., Page, A.N.: Translation of Manuale di economia politica (“Manual of political economy”), A.M. Kelley, (1971), ISBN 978-0-678-00881-2

  • Patterson, D., et al.: A case for Redundant Arrays of Inexpensive Disks (RAID). In: Proc. of the ACM SIGMOD ’88 (1988)

  • Plank, J.: The RAID-6 liberation codes. In: Proc. of the FAST ’08 (2008)

  • Reed, I., Solomon, G.: Polynomial codes over certain finite fields. J. Soc. Ind. Appl. Math. 8, 300–304 (1960)

    Article  MathSciNet  Google Scholar 

  • Sundermayer, M., Schlüter, R., Ney, H.: LSTM neural networks for language modeling. In: INTERSPEECH (2012)

  • Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: ACL (2015)

  • Wilkes, J., et al.: The HP autoRAID hierarchical storage system. ACM Trans. Comput. Syst. 14(1), 108–136 (1996)

    Article  Google Scholar 

  • Wu, S., Xu, Y., Li, Y., Zhu, Y.: POS: a popularity-based online scaling scheme for raid-structured storage systems. In: Proc. of the ICCD ’15 (2015). POS[26], STAR code[22], RS6[23], TIP code[24]

  • Wu, C., He, X.: GSR: a global stripe-based redistribution approach to accelerate RAID-5 scaling. In: 41st International Conference on Parallel Processing (2012)

  • Wu, C., He, X., Han, J., et al.: SDM: a stripe-based data migration scheme to improve the scalability of RAID-6. In: IEEE International Conference on Cluster Computing. IEEE (2012)

  • Wu, C., He, X., Li, J., et al.: Code 5-6: an efficient MDS array coding scheme to accelerate online RAID level migration. In: 2015 44th International Conference on Parallel Processing (ICPP). IEEE (2015)

  • Wu, S., Xu, Y., Li, Y., et al.: POS: a popularity-based online scaling scheme for RAID-structured storage systems. In: 2015 33rd IEEE International Conference on Computer Design (ICCD). IEEE Computer Society (2015)

  • Xie, X., Wu, C., Gu, J., Qiu, H., Li, J., Guo, M., He, X., Dong, Y., Zhao, Y.: AZ-Code: an efficient availability zone level erasure code to provide high fault tolerance in cloud storage systems. In: The 35th International Conference on Massive Storage Systems and Technology (MSST 2019). IEEE (2019)

  • Zhang, G., et al.: SLAS: an efficient approach to scaling round-robin striped volumes. ACM Trans. Storage 3(1), 1–39 (2007)

    Article  MathSciNet  Google Scholar 

  • Zhang, G., et al.: ALV: a new data redistribution approach to RAID-5 scaling. IEEE Trans. Comput. 59(3), 345–357 (2010)

    Article  MathSciNet  Google Scholar 

  • Zhang, G., Li, K., Wang, J., Zheng, W.: Accelerate rdp raid-6 scaling by reducing disk i/os and xor operations. IEEE Trans. Comput. 64(1), 32–44 (2015)

    Article  MathSciNet  Google Scholar 

  • Zhang, G., Wu, G., Lu, Y., et al.: Xscale: online X-code RAID-6 scaling using lightweight data reorganization. IEEE Trans. Parallel Distrib. Syst. 27(12), 1–1 (2016)

    Article  Google Scholar 

  • Zhang, Y., Wu, C., Li, J., Guo, M.: TIP-code: a three independent parity code to tolerate triple disk failures with optimal update complexity. In: Proc. of the IEEE/IFIP DSN ’15 (2015)

  • Zheng, W., Zhang, G.: FastScale: accelerate RAID scaling by minimizing data migration. In: Proc. of the USENIX FAST’11 (2011)

Download references

Acknowledgements

We thank anonymous reviewers for their insightful comments. This work is partially sponsored by the National Key R&D Program of China (No. 2018YFB0105203), the Natural Science Foundation of China (NSFC) (No. 61972246), and the Natural Science Foundation of Shanghai (No. 18ZR1418500).

Author information

Author notes

  1. Zhehan Lin and Hanchen Guo have the same contribution on this paper.

Authors and Affiliations

  1. Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, China

    Zhehan Lin, Hanchen Guo & Chentao Wu

Authors
  1. Zhehan Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanchen Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chentao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chentao Wu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, Z., Guo, H. & Wu, C. AIR: an approximate intelligent redistribution approach to accelerate RAID scaling. CCF Trans. HPC 2, 50–66 (2020). https://doi.org/10.1007/s42514-020-00021-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42514-020-00021-0

Keywords

Search

Navigation