skip to main content
research-article

HPDA: A hybrid parity-based disk array for enhanced performance and reliability

Published: 24 February 2012 Publication History

Abstract

Flash-based Solid State Drive (SSD) has been productively shipped and deployed in large scale storage systems. However, a single flash-based SSD cannot satisfy the capacity, performance and reliability requirements of the modern storage systems that support increasingly demanding data-intensive computing applications. Applying RAID schemes to SSDs to meet these requirements, while a logical and viable solution, faces many challenges. In this article, we propose a Hybrid Parity-based Disk Array architecture (short for HPDA), which combines a group of SSDs and two hard disk drives (HDDs) to improve the performance and reliability of SSD-based storage systems. In HPDA, the SSDs (data disks) and part of one HDD (parity disk) compose a RAID4 disk array. Meanwhile, a second HDD and the free space of the parity disk are mirrored to form a RAID1-style write buffer that temporarily absorbs the small write requests and acts as a surrogate set during recovery when a disk fails. The write data is reclaimed to the data disks during the lightly loaded or idle periods of the system. Reliability analysis shows that the reliability of HPDA, in terms of MTTDL (Mean Time To Data Loss), is better than that of either pure HDD-based or SSD-based disk array. Our prototype implementation of HPDA and the performance evaluations show that HPDA significantly outperforms either HDD-based or SSD-based disk array.

References

[1]
Agrawal, N., Bolosky, W. J., Douceur, J. R., and Lorch, J. R. 2007. A five-year study of file-system metadata. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST'07). USENIX, Berkeley, CA, 31--45.
[2]
Agrawal, N., Prabhakaran, V., Wobber, T., Davis, J., Manasse, M., and Panigrahy, R. 2008. Design tradeoffs for SSD performance. In Proceedings of the USENIX Annual Technical Conference (USENIX'08). USENIX, Berkeley, CA, 57--70.
[3]
Andersen, D. G., Franklin, J., Kaminsky, M., Phanishayee, A., Tan, L., and Vasudevan, V. 2009. FAWN: A Fast Array of Wimpy Nodes. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles (SOSP'09). 1--14.
[4]
Balakrishnan, M., Kadav, A., Prabhakaran, V., and Malkhi, D. 2010. Differential RAID: Rethinking RAID for SSD reliability. In Proceedings of the 5th ACM European Conference on Computer Systems (EuroSys'10). ACM, New York, 15--26.
[5]
Chen, F., Koufaty, D. A., and Zhang, X. 2009. Understanding intrinsic characteristics and system implications of flash memory based solid state drives. In Proceedings of the ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (SIGMETRICS/Performance'09). 181--192.
[6]
Chen, F., Luo, T., and Zhang, X. 2011. CAFTL: A Content-Aware Flash Translation Layer enhancing the lifespan of flash memory based solid state drives. In Proceedings of the 9th USENIX Conference on File and Storage Technologies (FAST'11). USENIX, Berkeley, CA, 77--90.
[7]
Chen, Y., Hsu, W., and Young, H. 2000. Logging RAID---An approach to fast, reliable, and low-cost disk arrays. In Proceedings of the 6th International Euro-Par Conference on Parallel Processing (Euro-Par'00). 1302--1311.
[8]
Dirik, C. and Jacob, B. 2009. The performance of PC solid-state disks as a function of bandwidth, concurrency, device architecture, and system organization. In Proceedings of the 36th International Symposium on Computer Architecture (ISCA'09).
[9]
Elerath, J. G. and Pecht, M. 2007. Enhanced reliability modeling of raid storage systems. In Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07). 175--184.
[10]
Golding, R., Bosch, P., and Staelin, C. 1995. Idleness is not sloth. In Proceedings of the USENIX Technical Conference (USENIX'95). USENIX, Berkeley, CA, 201--212.
[11]
Greenan, K. M., Long, D. D. E., Miller, E. L., Schwarz, T. J. E., and Wildani, A. 2009. Building flexible, fault-tolerant flash-based storage systems. In Proceedings of the 5th Workshop on Hot Topics in System Dependability (HotDep'09).
[12]
Greenan, K. M., Plank, J. S., and Wylie, J. J. 2010. Mean time to meaningless: MTTDL, Markov models, and storage system reliability. In Proceedings of the Workshop on Hot Topics in Storage and File Systems (HotStorage'10). USENIX, Berkeley, CA.
[13]
Gupta, A., Kim, Y., and Urgaonkar, B. 2009. DFTL: A flash translation layer employing demand-based selective caching of page-level address mappings. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'09). 229--240.
[14]
Gupta, A., Pisolkar, R., Urgaonkar, B., and Sivasubramaniam, A. 2011. Leveraging value locality in optimizing NAND flash-based SSDs. In Proceedings of the 9th USENIX Conference on File and Storage Technologies (FAST'11). USENIX, Berkeley, CA, 91--104.
[15]
Hu, Y. and Yang, Q. 1996. DCD---Disk caching disk: A new approach for boosting I/O performance. In Proceedings of the 23rd Annual International Symposium on Computer Architecture (ISCA'96). 169--178.
[16]
Hu, Y., Yang, Q., and Nightingale, T. 1999. RAPID-Cache---A Reliable and Inexpensive Write Cache for Disk I/O Systems. In Proceedings of the 5th International Symposium on High Performance Computer Architecture (HPCA'99). 204--213.
[17]
Im, S. and Shin, D. 2011. Flash-Aware RAID techniques for dependable and high-performance flash memory SSD. IEEE Trans. Comput. 60, 1, 80--92.
[18]
Intel Technical Report. 1998. Understanding the flash translation layer (FTL) specification.
[19]
IOmeter. http://sourceforge.net/projects/iometer.
[20]
Kadav, A., Balakrishnan, M., Prabhakaran, V., and Malkhi, D. 2009. Differential RAID: Rethinking RAID for SSD reliability. In Proceedings of the Workshop on Hot Topics in Storage and File Systems (HotStorage'09).
[21]
Kim, H. and Ahn, S. 2008. BPLRU: A buffer management scheme for improving random writes in flash storage. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST'08). USENIX, Berkeley, CA, 239--252.
[22]
Kim, Y., Gupta, A., and Urgaonkar, B. 2008. MixedStore: An enterprise-scale storage system combining solid-state and hard disk drives. Tech. rep., Department of Computer Science and Engineering, The Pennsylvania State University.
[23]
Kim, Y., Oral, S., Shipman, G., Lee, J., Dillow, D., and Wang, F. 2011. Harmonia: A globally coordinated garbage collector for arrays of solid-state drives. In Proceedings of the IEEE Symposium on Massive Storage Systems and Technologies (MSST'11). IEEE.
[24]
Koltsidas, I. and Viglas, S. D. 2008. Flashing up the storage layer. In Proceedings of the 34th International Conference on Very Large Data Bases (VLDB'08). 514--525.
[25]
Mao, B., Feng, D., Jiang, H., Wu, S., Chen, J., and Zeng, L. 2008. GRAID: A green RAID storage architecture with improved energy efficiency and reliability. In Proceedings of the 16th Annual Meeting of the IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS'08). 113--120.
[26]
Menon, J. 1995. A performance comparison of RAID-5 and log-structured arrays. In Proceedings of the 4th International Symposium on High Performance Distributed Computing (HPDC'95). ACM, New York.
[27]
Mi, N., Casale, G., Cherkasova, L., and Smirni, E. 2008. Burstiness in multi-tier applications: Symptoms, causes, and new models. In Proceedings of the 9th ACM/IFIP/USENIX International Middleware Conference (Middleware'08). ACM, New York, 265--286.
[28]
Mogi, K. and Kitsuregawa, M. 1996. Hot mirroring: a method of hiding parity update penalty and degradation during rebuilds for RAID5. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'96). ACM, New York, 183--194.
[29]
Narayanan, D., Thereska, E., Donnelly, A., Elnikety, S., and Rowstron, A. 2009. Migrating server storage to SSDs: Analysis of tradeoffs. In Proceedings of the 4th European Conference on Computer Systems (EuroSys'09).
[30]
OLTP Trace. http://traces.cs.umass.edu/index.php/Storage/Storage.
[31]
Patterson, D., Gibson, G., and Katz, R. 1988. A case for redundant arrays of inexpensive disks (RAID). In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'88). ACM, New York, 109--116.
[32]
Pâris, J., Amer, A., and Long, D. D. E. 2009. Using storage class memories to increase the reliability of two-dimensional RAID arrays. In Proceedings of the 17th Annual Meeting of the IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS'09). 1--8.
[33]
Riska, A. and Riedel, E. 2006. Disk drive level workload characterization. In Proceedings of the USENIX Annual Technical Conference (USENIX'06). USENIX, Berkeley, CA, 97--102.
[34]
Rosenblum, M. and Ousterhout, J. K. 1992. The design and implementation of a log-structured file system. ACM Trans. Comput. Syst. 10, 1, 26--52.
[35]
Ruemmler, C. and Wilkes, J. 1993. UNIX disk access patterns. In Proceedings of the USENIX Winter Technical Conference (USENIX'93). USENIX, Berkeley, CA, 405--420.
[36]
Samsung Report. http://news.cnet.com/8301-13924_3-9876557-64.html.
[37]
SanDisk SSD. http://www.sandisk.com/.
[38]
Savage, S. and Wilkes, J. 1996. AFRAID: A Frequently Redundant Array of Independent Disks. In Proceedings of the USENIX Annual Technical Conference (USENIX'96). USENIX, Berkeley, CA, 27--39.
[39]
Schroeder, B. and Gibson, G. A. 2007. Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you? In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST'07). USENIX, Berkeley, CA, 1--16.
[40]
Soundararajan, G., Prabhakaran, V., Balakrishnan, M., and Wobber, T. 2010. Extending SSD lifetimes with disk-based write caches. In Proceedings of the 8th USENIX Conference on File and Storage Technologies (FAST'10). USENIX, Berkeley, CA, 101--114.
[41]
SSSI, S. http://www.snia.org/forums/sssi/.
[42]
Stodolsky, D., Gibson, G., and Holland, M. 1993. Parity logging overcoming the small write problem in redundant disk arrays. In Proceedings of the 20th Annual International Symposium on Computer Architecture (ISCA'93). 64--75.
[43]
Storer, M. W., Greenan, K. M., Miller, E. L., and Voruganti, K. 2008. Pergamum: Replacing tape with energy efficient, reliable, disk-based archival storage. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST'08). USENIX, Berkeley, CA, 1--16.
[44]
Tian, L., Feng, D., Jiang, H., Zhou, K., Zeng, L., Chen, J., Wang, Z., and Song, Z. 2007. PRO: A popularity-based multi-threaded reconstruction optimization for RAID-structured storage systems. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST'07). USENIX, Berkeley, CA, 277--290.
[45]
Uysal, M., Merchantand, A., and Alvarez, G. A. 2003. Using MEMS-based storage in disk arrays. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies (FAST'03). USENIX, Berkeley, CA, 89--102.
[46]
Wilkes, J., Golding, R., Staelin, C., and Sullivan, T. 1995. The HP AutoRAID hierarchical storage system. Oper. Syst. Rev. 29, 5, 96--108.
[47]
Wu, S., Jiang, H., Feng, D., Tian, L., and Mao, B. 2009. WorkOut: I/O workload outsourcing for boosting RAID reconstruction performance. In Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST'09). USENIX, Berkeley, CA, 239--252.
[48]
Xie, T. and Sun, Y. 2010. Dynamic data reallocation in hybrid disk arrays. IEEE Trans. Parall. Distrib. Syst. 21, 9, 1330--1341.
[49]
Zhu, Q., Chen, Z., Tan, L., Zhou, Y., Keeton, K., and Wilkes, J. 2005. Hibernator: Helping disk arrays sleep through the winter. In Proceedings of the ACM SIGOPS 20th Symposium on Operating Systems Principles (SOSP'05). ACM, New York, 177--190.

Cited By

View all
  • (2024)Flash-oriented Coded Storage: Research Status and Future DirectionsACM Transactions on Storage10.1145/370899521:1(1-37)Online publication date: 19-Dec-2024
  • (2024)HybRAID: A High-Performance Hybrid RAID Storage Architecture for Write-Intensive Applications in All-Flash Storage SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.342933635:12(2608-2623)Online publication date: Dec-2024
  • (2024)AquaSonic: Acoustic Manipulation of Underwater Data Center Operations and Resource Management2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00201(331-349)Online publication date: 19-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage
ACM Transactions on Storage  Volume 8, Issue 1
February 2012
92 pages
ISSN:1553-3077
EISSN:1553-3093
DOI:10.1145/2093139
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 February 2012
Accepted: 01 August 2011
Revised: 01 July 2011
Received: 01 December 2010
Published in TOS Volume 8, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. RAID
  2. SSD
  3. Storage systems
  4. performance
  5. reliability

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Flash-oriented Coded Storage: Research Status and Future DirectionsACM Transactions on Storage10.1145/370899521:1(1-37)Online publication date: 19-Dec-2024
  • (2024)HybRAID: A High-Performance Hybrid RAID Storage Architecture for Write-Intensive Applications in All-Flash Storage SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.342933635:12(2608-2623)Online publication date: Dec-2024
  • (2024)AquaSonic: Acoustic Manipulation of Underwater Data Center Operations and Resource Management2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00201(331-349)Online publication date: 19-May-2024
  • (2024)Performance Analysis of Kubernetes Job Scheduling Model Based on Queuing Theory2024 IEEE 2nd International Conference on Sensors, Electronics and Computer Engineering (ICSECE)10.1109/ICSECE61636.2024.10729560(592-595)Online publication date: 29-Aug-2024
  • (2023)Realizing Strong Determinism Contract on Log-Structured Merge Key-Value StoresACM Transactions on Storage10.1145/358269519:2(1-29)Online publication date: 25-Mar-2023
  • (2022)Mlog: Multi-log Write Buffer upon Ultra-fast SSD RAIDProceedings of the 51st International Conference on Parallel Processing10.1145/3545008.3545034(1-11)Online publication date: 29-Aug-2022
  • (2022)Exploration and Exploitation for Buffer-Controlled HDD-Writes for SSD-HDD Hybrid Storage ServerACM Transactions on Storage10.1145/346541018:1(1-29)Online publication date: 29-Jan-2022
  • (2022)I/O Workload Management for All-Flash Datacenter Storage Systems Based on Total Cost of OwnershipIEEE Transactions on Big Data10.1109/TBDATA.2018.28711148:2(332-345)Online publication date: 1-Apr-2022
  • (2020)BCWProceedings of the 18th USENIX Conference on File and Storage Technologies10.5555/3386691.3386716(253-266)Online publication date: 24-Feb-2020
  • (2020)Patch-based Data Management for Dual-copy Buffers in RAID-enabled SSDsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2020.3012252(1-1)Online publication date: 2020
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media