Skip to main content
SpringerLink
Log in

Adaptive Sector Grouping to Reduce False Sharing in Distributed RAID

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Distributed redundant array of inexpensive disks (RAID) is often embedded in a cluster architecture. In a centralized RAID subsystem, the false sharing problem does not exist, because the disk array allows only mutually exclusive access by one user at a time. However, the problem does exist in a distributed RAID architecture, because multiple accesses may occur simultaneously in a distributed environment. This problem will seriously limit the effectiveness of collective I/O operations in network-based, cluster computing. Traditional accesses to disks in a RAID are done at block level. The block granularity is large, say 32 KB, often resulting in false sharing among fragments in the block. The false sharing problem becomes worse when the block size or the stripe unit becomes too large. To solve this problem, we propose an adaptive sector grouping approach to accessing a distributed RAID. Each sector has a fine grain of 512 B. Multiple sectors are grouped together to match with the data block size. The grouped sector has a variable size that can be adaptively adjusted by software. Benchmark experiments reveal the positive effects of this adaptive access scheme on the performance of a RAID. Our scheme can reduce the collective I/O access time without increasing the buffer size. Both theoretical analysis and experimental results demonstrate the performance gain in using grouped sectors for fast access of distributed RAID.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. C. Amza, A.L. Cox, S.D. Warkadas, P. Kelehr, H. Lu, R. Rajamony, W. Yu and W. Zwaenepoel, TreadMarks: shared memory computing on networks of workstations, IEEE Computer 29(2) (1996) 18–28.

    Google Scholar 

  2. T. Anderson, M. Dahlin, J. Neefe, D. Patterson, D. Roselli and R. Wang, Serverless network file systems, ACM Transactions on Computer Systems (January 1996) 41–79.

  3. W.J. Bolosky and M.L. Scott, False sharing and its effect on shared memory performance, in: Proceedings of the USENIX Symposium on Experiences with Distributed and Multiprocessor Systems (September 1993) pp. 57–72.

  4. P. Brezany, Input/Output Intensive Massively Parallel Computing: Language Support, Automatic Parallelization, Advanced Optimization, and Runtime Systems, Lecture Notes in Computer Science, Vol. 1220 (Springer, 1997).

  5. R. Buyya, ed., High Performance Cluster Computing - Architectures and Systems (Prentice Hall, Englewood Cliffs, NJ, 1999).

  6. L.F. Cabrera and D.D.E. Long, Swift: Using distributed disk striping to provide high I/O data rates, in: Proceedings of USENIX Computing Systems (Fall 1991) pp. 405–433.

  7. P. Cao, S.B. Lim, S. Venkataraman and J. Wilkes, The TickerTAIP parallel RAID architecture, ACM Transactions on Computer System 12(3) (August 1994) 236–269.

    Google Scholar 

  8. P.M. Chen, E.K. Lee, G.A. Gibson, R.H. Katz and D.A. Patterson, RAID: High-performance, reliable secondary storage, ACM Computing Surveys 26(2) (June 1994) 145–185.

    Google Scholar 

  9. T. Clark and T. Clark, Designing Storage Area Networks (Addison-Wesley, 1999).

  10. S.J. Eggers and T.E. Jeremiassen, Eliminating false sharing, in: Proceedings of the 1991 International Conference on Parallel Processing, Vol. 1 (August 1991) pp. 377–381.

    Google Scholar 

  11. M. Farley and T.A. May, Building Storage Networks (McGraw-Hill, 2000).

  12. G. Gibson, Redundant Disk Arrays: Reliable, Parallel Secondary Storage (MIT Press, 1992).

  13. G.A. Gibson, D.F. Nagle, K. Amiri, F.W. Chang, E.M. Feinberg, H. Gobioff, C. Lee, B. Ozceri, E. Riedel, D. Rochberg and J. Zelenka, File server scaling with network-attached secure disks, in: Proceedings of the ACM Sigmetrics'97 (June 1997).

  14. G. Gibson, D. Nagle, K. Amiri, F. Chang, H. Gobioff, E. Riedel, D. Rochberg and J. Zelenka, A cost-effective, high-bandwidth storage architecture, in: Proc. of the 8th Conf. on Architectural Support for Programming Langagues and Operating Systems (1998).

  15. J.H. Hartman, I. Murdock and T. Spalink, The swarm scalable storage system, in: Proceedings of the 19th IEEE International Conference on Distributed Computing Systems (ICDCS’ 99) (June 1999).

  16. K. Hwang, H. Jin and R. Ho, RAID-x: A new distributed disk array for I/O-centric cluster computing, in: Proceedings of 9th IEEE International Symposium on High Performance Distributed Computing (HPDC-9), Pittsburgh, PA (1–4 August 2000) pp. 279–286.

  17. K. Hwang, H. Jin, E. Chow, C.-L. Wang and Z. Xu, Designing SSI clusters with hierarchical checkpointing and single I/O space, IEEE Concurrency 7(1) (1999) 60–69.

    Google Scholar 

  18. K. Hwang and Z. Xu, Scalable Parallel Computing: Technology, Architecture, Programming (WCB/McGraw-Hill, 1998).

  19. R.L. Hyde and B.D. Fleisch, Degenerate sharing, in: Proceedings of the 1994 International Conference on Parallel Processing, Vol. 1 (1994) pp. 267–270.

    Google Scholar 

  20. H. Jin, J. He, Q. Chen and K. Hwang, Grouped RAID accesses to reduce false sharing effect in clusters with single I/O space, Proceedings of 2nd International Symposium on High Performance Computing, Lecture Notes in Computer Science, Vol. 1615 (Springer, 1999).

  21. H. Jin and K. Hwang, False sharing problems in distributed RAIDs, Proceedings of 1999 ACM Symposium on Applied Computing, San Antonio, Texas (February 1999) pp. 461–465.

  22. H. Jin and K. Hwang, Performance effect analysis of false sharing problem in clusters with single I/O space, in: Proc. of 1999 Int. Conf. on Parallel and Distributed Processing Techniques and Applications, Vol. VI, Las Vegas, Nevada (July 1999) pp. 2788–2794.

    Google Scholar 

  23. H. Jin, X. Zhou and P. Cheng, Track-based improved LRU algorithm for disk array, in: Proceedings of 15th IMACS World Congress on Scientific Computation Modeling and Applied Mathematics, Vol. IV (Berlin, 1997) pp. 583–588.

    Google Scholar 

  24. R.H. Katz, Network-attached storage systems, in: Proceedings of Scalable High Performance Computing Conference (April 1992) pp. 68–75.

  25. V. Khera, R.P. Larowe and C.S. Ellis, An architecture-independent analysis of false sharing, TR 93–006, Computer Science Department, Duke University (October 1993).

  26. V. Khera, Factors affecting false sharing on page-granularity cachecoherent shared-memory multiprocessors, Ph.D. Dissertation, CS-1994–37, Duke University (December 1994).

  27. E. Lee and R. Katz, The performance of parity placement in disk arrays, IEEE Transactions on Computers C-42(6) (1993) 651–664.

    Google Scholar 

  28. E.K. Lee and C.A. Thekkath, Petal: Distributed virtual disks, in: Proceedings of the 7th International Conference on Architectural Support for Programming Languages and Operating Systems, Cambridge, MA (October 1996) pp. 84–92.

  29. K. Li, IVY: A shared virtual memory system for parallel computing, in: Proceedings of 1988 International Conference on Parallel Processing, Vol. II (1988) pp. 94–101.

    Google Scholar 

  30. D.A. Menascé, O.I. Pentakalos and Y. Yesha, An analytic model of hierarchical mass storage systems with network-attached storage devices, in: Proceedings of the ACM International Conference on Measurement and Modeling of Computer Systems (1996) pp. 180–189.

  31. S. Nakamura, H. Minemura, T. Yamaguchi, H. Shimizu, T. Watanabe and T. Mizuno, Distributed RAID style video server, IEICE Transactions on Communication E79–B(8) (August 1996) 1030–1037.

    Google Scholar 

  32. G.F. Pfister, The varieties of single system image, in: Proceedings of IEEE Workshop on Advances in Parallel and Distributed System (IEEE CS Press, 1993) pp. 59–63.

  33. Quantum Co.: Storage Basics. http://www.quantum.com/ src/storage_basics/

  34. E. Riedel, Active disks: Remote execution for network-attached storage, Ph.D. Dissertation, CMU-CS-99–177 (November 1999).

  35. E. Riedel and G.A. Gibson, Understanding customer dissatisfaction with underutilized distributed file servers, in: Proceedings of the 5th NASA Goddard Space Flight Center Conference on Mass Storage Systems and Technologies (September 1996).

  36. E. Riedel, G.A. Gibson and C. Faloutsos, Active storage for largescale data mining and multimedia, in: Proceedings of the 24th International Conference on Very Large Databases (August 1998).

  37. M. Stonebraker and G.A. Schloss, Distributed RAID - a new multiple copy algorithm, in: Proceedings of the 6th International Conference on Data Engineering (February 1990) pp. 430–437.

  38. N. Talagala, S. Asami, D. Patterson and K. Lutz, Tertiary disk: Large scale distributed storage, UCB Technical Report No. UCB//CSD-98–989.

  39. J.M. Toigo and M.R. Toigo, The Holy Grail of Data Storage Management (Prentice Hall, 1999).

  40. J. Torrellas, M.S. Lam and J.L. Hennessy, False sharing and spatial locality in multiprocessor caches, IEEE Transactions on Computers C-43(6) (June 1994) 651–663.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer, Hvazhong University of Science and Technology, Wuhan, 430074, China

    Hai Jin

  2. Internet and Cluster Computing Laboratory, University of Southern California, Los Angeles, CA, 90089, USA

    Kai Hwang

Authors
  1. Hai Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jin, H., Hwang, K. Adaptive Sector Grouping to Reduce False Sharing in Distributed RAID. Cluster Computing 4, 133–143 (2001). https://doi.org/10.1023/A:1011468915637

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011468915637

Search

Navigation