Skip to main content
Log in

Architectures and algorithms for on-line failure recovery in redundant disk arrays

  • Published:
Distributed and Parallel Databases Aims and scope Submit manuscript

Abstract

The performance of traditional RAID Level 5 arrays is, for many applications, unacceptably poor while one of its constituent disks is non-functional. This paper describes and evaluates mechanisms by which this disk array failure-recovery performance can be improved. The two key issues addressed are thedata layout, the mapping by which data and parity blocks are assigned to physical disk blocks in an array, and thereconstruction algorithm, which is the technique used to recover data that is lost when a component disk fails.

The data layout techniques this paper investigates are instantiations of thedeclustered parity organization, a derivative of RAID Level 5 that allows a system to trade some of its data capacity for improved failure-recovery performance. We show that our instantiations of parity declustering improve the failure-mode performance of an array significantly, and that a parity-declustered architecture is preferable to an equivalent-size multiple-group RAID Level 5 organization in environments where failure-recovery performance is important. The presented analyses also include comparisons to a RAID Level 1 (mirrored disks) approach.

With respect to reconstruction algorithms, this paper describes and briefly evaluates two alternatives,stripeoriented reconstruction anddisk-oriented reconstruction, and establishes that the latter is preferable as it provides faster reconstruction. The paper then revisits a set of previously-proposed reconstruction optimizations, evaluating their efficacy when used in conjunction with the disk-oriented algorithm. The paper concludes with a section on the reliability versus capacity trade-off that must be addressed when designing large arrays.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. American National Standard for Information Systems—Small Computer System Interface (SCSI), ANSI X3.132-1986, New York NY, 1986.

  2. American National Standard for Information Systems—High Performance Parallel Interface—Mechanical, Electrical, and Signalling Protocol Specification, ANSI X3.183-1991, New York NY, 1991.

  3. Arulpragasam, J. and Swarz, R., “A Design for State Preservation on Storage Unit Failure,”Proceedings of the International Symposium on Fault Tolerant Computing, 1980, pp. 47–52.

  4. Bitton, D. and Gray, J., “Disk Shadowing,”Proceedings of the 14th Conference on Very Large Data Bases, 1988, pp. 331–338.

  5. Cao, P., Lim, S.B., Venkataraman S. and Wilkes J., “The Ticker TAIP parallel RAID architecture,”Proceedings of the International Symposium on Computer Architecture, 1993, pp. 52–63.

  6. Chee, Y.M., Colbourn, C., Kreher D., “Simplet-designs withv ≤ 30,”Ars Combinatoria, v. 29, 1990.

  7. Chen, P., et al., “An Evaluation of Redundant Arrays of Disks using an Amdahl 5890,”Proceedings of the Conference on Measurement and Modeling of Computer Systems, 1990, pp. 74–85.

  8. Chen, P. and Patterson, D., “Maximizing Performance in a Striped Disk Array,”Proceedings of International Symposium on Computer Architecture, 1990, pp. 322–331.

  9. Copeland, G. and Keller, T., “A Comparison of High-Availability Media Recovery Techniques,”Proceedings of the ACM Conference on Management of Data, 1989, pp. 98–109.

  10. Dibble, P., “A Parallel Interleaved File System,” University of Rochester Technical Report 334, 1990.

  11. Fibre Channel—Physical Layer, ANSI X3T9.3 Working Document, Revision 2.1, May 1991.

  12. Gibson, G.,Redundant Disk Arrays: Reliable, Parallel Secondary Storage, MIT Press, 1992.

  13. Gibson, G. and Patterson, D., “Designing Disk Arrays for High Data Reliability,”Journal of Parallel and Distributed Computing, Vol. 17, 1993, pp. 4–27.

    Google Scholar 

  14. Gray, G., Horst, B. and Walker, M., “Parity Striping of Disc Arrays: Low-Cost Reliable Storage with Acceptable Throughput,”Proceedings of the Conference on Very Large Data Bases, 1990, pp. 148–160.

  15. Hall M.,Combinatorial Theory (2nd Edition), Wiley-Interscience, 1986.

  16. Hanani, H., “Balanced Incomplete Block Designs and Related Designs,”Discrete Mathematics, v. 11, 1975.

  17. Holland, M. and Gibson, G., “Parity Declustering for Continuous Operation in Redundant Disk Arrays,”Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems, 1992, pp. 23–25.

  18. Holland, M., Gibson, G. and Siewiorek, D., “Fast, On-Line Failure Recovery in Redundant Disk Arrays,”Proceedings of the International Symposium on Fault-Tolerant Computing, 1993, pp. 422–431.

  19. Hou, R., Menon, J. and Patt, Y., “Balancing I/O Response Time and Disk Rebuild Time in a RAID5 Disk Array,”Proceedings of the Hawaii International Conference on Systems Sciences, 1993, pp. 70–79.

  20. Hsiao, H.-I. and DeWitt, D.J., “A Performance Study of Three High-Availability Data Replication Strategies,”Proceedings of the International Conference on Parallel and Distributed Information Systems, 1991, pp. 18–28.

  21. IBM Corporation, IBM 0661 Disk Drive Product Description, Model 370, First Edition, Low End Storage Products, 504/114-2, 1989.

  22. Proposed IEEE Standard 802.6—Distributed Queue Dual Bus (DQDB)—Metropolitan Area Network, Draft D7, IEEE 802.6 Working Group, 1989.

  23. IEEE High Performance Serial Bus Specification, P1394/Draft 6.2v0, New York, NY, June, 1993.

  24. Katz, R., et al., “A Project on High Performance I/O Subsystems,”ACM Computer Architecture News, Vol. 17(5), 1989, pp. 24–31.

    Google Scholar 

  25. Katzman, J., “System Architecture for Nonstop Computing,”Proceedings of the Computer Society International Conference (COMPCON 77), 1977.

  26. Kim, M., “Synchronized Disk Interleaving,”IEEE Transactions on Computers, Vol. 35 (11), 1986, pp. 978–988.

    Google Scholar 

  27. Lee, E., “Software and Performance Issues in the Implementation of a RAID Prototype,” University of California Technical Report UCB/CSD 90/573, 1990.

  28. Lee, E. and Katz, R., “Performance Consequences of Parity Placement in Disk Arrays,”Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems, 1991, pp. 190–199.

  29. Livny, M., Khoshafian, S. and Boral H., “Multi-disk Management Algorithms,”Proceedings of the ACM Conference on Measurement and Modeling of Computer Systems, 1987, pp. 69–77.

  30. Mathon, R. and Rosa, A., “Tables of Parameters of BIBDs withr ≤ 41 Including Existence, Enumeration and Resolvability Results: An Update,”Ars Combinatoria, v. 30, 1990.

  31. Menon, J. and Kasson, J., “Methods for Improved Update Performance of Disk Arrays,”Proceedings of the Hawaii International Conference on System Sciences, 1992, pp. 74–83.

  32. Menon, J. and Mattson, D., “Comparison of Sparing Alternative for Disk Arrays,”Proceedings of the International Symposium on Computer Architecture, 1992, pp. 318–329.

  33. Menon, J. and Cortney, J., “The Architecture of a Fault-Tolerant Cached RAID Controller,”Proceedings of the International Symposium on Computer Architecture, 1993, pp. 76–86.

  34. Merchant, A. and Yu, P., “Design and Modeling of Clustered RAID,”Proceedings of the International Symposium on Fault-Tolerant Computing, 1992, pp. 140–149.

  35. Mills, W.H. and Mullin, R.C., “Coverings and Packings,” Chapter 9 inContemporary Design Theory: A Collection of Surveys, John Wiley & Sons, Inc., 1992, pp. 371–399.

  36. Muntz, R. and Lui, J., “Performance Analysis of Disk Arrays Under Failure,”Proceedings of the Conference on Very Large Data Bases, 1990, pp. 162–173.

  37. Ng, S. and Mattson, R., “Maintaining Good Performance in Disk Arrays During Failure Via Uniform Parity Group Distribution,”Proceedings of the First International Symposium on High-Performance Distributed Computing, 1992, pp. 260–269.

  38. Park, A. and Balasubramanian K., “Providing Fault Tolerance in Parallel Secondary Storage Systems,” Princeton University Technical Report CS-TR-057-86, 1986.

  39. Patterson, D., Gibson, G. and Katz, R., “A Case for Redundant Arrays of Inexpensive Disks (RAID),”Proceedings of the Conference on Management of Data, 1988, pp. 109–116.

  40. Peterson, W. and Weldon, Jr., E.,Error-Correcting Codes, second edition, MIT Press, 1972.

  41. The RAIDBook, A Source Book for RAID Technology, published by the RAID Advisory Board, Lino Lakes, Minnesota, 1993.

  42. Ramakrishnan, K., Biswas, P. and Karedla, R., “Analysis of File I/O Traces in Commercial Computing Environments,”Proceedings of the Conference on Measurement and Modeling of Computer Systems, 1992, pp. 78–90.

  43. Reddy, A.L.N. and Bannerjee, P., “Gracefully Degradable Disk Arrays,”Proceedings of the International Symposium on Fault-Tolerant Computing, 1991, pp. 401–408.

  44. Rosenblum, M. and Ousterhout, J., “The Design and Implementation of a Log-Structured File System,”Proceedings of the Symposium on Operating System Principles, 1991, pp. 1–15.

  45. Schulze, M., Gibson, G., Katz, R. and Patterson D., “How Reliable is a RAID?,”Proceedings of COMPCON, 1989, pp. 118–123.

  46. Stodolsky, D., Gibson, G. and Holland, M., “Parity Logging: Overcoming the Small-Write Problem in Redundant Disk Arrays,”Proceedings of the International Symposium on Computer Architecture, 1993, pp. 64–75.

  47. “DBC/1012 Data Base Computer System Manual, Release 1.3,” C10-0001-01,Teradata Corporation, 1985.

  48. The TPC-A Benchmark: A Standard Specification, Transaction Processing Performance Council, 1989.

Download references

Author information

Authors and Affiliations

Authors

Additional information

Portions of this material are drawn from papers at the 5th Conference on Architectural Support for Programming Languages and Operating Systems, 1992, and at the 23rd Symposium on Fault-Tolerant Computing, 1993. The work was supported by the National Science Foundation under grant number ECD-8907068, by the Defense Advanced Research Project Agency monitored by ARPA/CMO under contract MDA972-90-C-0035, and by an IBM Graduate Fellowship.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Holland, M., Gibson, G.A. & Siewiorek, D.P. Architectures and algorithms for on-line failure recovery in redundant disk arrays. Distrib Parallel Databases 2, 295–335 (1994). https://doi.org/10.1007/BF01266332

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01266332

Keywords

Navigation