skip to main content
10.1145/2485732.2485737acmconferencesArticle/Chapter ViewAbstractPublication PagessystorConference Proceedingsconference-collections
research-article

Fuzzy adaptive control for heterogeneous tasks in high-performance storage systems

Published: 30 June 2013 Publication History

Abstract

Beyond handling user reads and writes, storage systems execute multiple background tasks of various types, such as reconstruction of missing parity data and defragmentation. The resources of the system must be divided between user loads and internal tasks using a specific policy.
This work describes Fuzzy Adaptive Control -- an innovative mechanism for sharing resources among various types of highly-variable loads. The new approach uses throughput as the task progress indicator avoiding assumptions about task properties such as resource consumption or handling process. It makes this new technique particularly well suited for complex systems where defining an accurate task model is difficult.
The Fuzzy Adaptive Control is evaluated on resource division between user loads and background tasks in HYDRAstor -- a commercial high-performance distributed secondary storage system. The presented mechanism is compared with fair queuing variants and is shown to be more stable in the case of irregular workload. The evaluation proves that our approach is responsive to changing load conditions and ensures high resource utilization.

References

[1]
D. R. Bobbarjung, S. Jagannathan, and C. Dubnicki. Improving duplicate elimination in storage systems. Trans. Storage, 2(4):424--448, 2006.
[2]
J. Bruno, J. Brustoloni, E. Gabber, B. Ozden, and A. Silberschatz. Disk scheduling with quality of service guarantees. In ICMCS '99, page 400, 1999.
[3]
H. M. Chaskar and U. Madhow. Fair scheduling with tunable latency: a round-robin approach. IEEE/ACM Trans. Netw., 11(4):592--601, 2003.
[4]
Y. Diao, C. M. Garcia-arellano, J. L. Hellerstein, S. S. Lightstone, S. S. Parekh, A. J. Storm, and M. Surendra. Systems and methods for providing constrained optimization using adaptive regulatory control, December 2005. US patent application no 20050268063.
[5]
J. R. Douceur and W. J. Bolosky. Progress-based regulation of low-importance processes. SIGOPS Oper. Syst. Rev., 33(5):247--260, Dec. 1999.
[6]
C. Dubnicki, L. Gryz, L. Heldt, M. Kaczmarczyk, W. Kilian, P. Strzelczak, J. Szczepkowski, C. Ungureanu, and M. Welnicki. HYDRAstor: a Scalable Secondary Storage. In FAST '09, 2009.
[7]
L. Eggert and J. D. Touch. Idletime scheduling with preemption intervals, proceedings of the twentieth acm symposium on operating systems principles, brighton,. In Twentieth ACM symposium on Operating systems principles, pages 249--262, 2005.
[8]
P. Goyal, H. M. Vin, and H. Chen. Start-time fair queueing: a scheduling algorithm for integrated services packet switching networks. In SIGCOMM '96, pages 157--168, 1996.
[9]
S. D. Gribble, G. S. Manku, D. Roselli, E. A. Brewer, T. J. Gibson, and E. L. Miller. Self-similarity in file systems. SIGMETRICS Perform. Eval. Rev., 26(1):141--150, 1998.
[10]
A. Gulati and I. Ahmad. Towards distributed storage resource management using flow control. SIGOPS Oper. Syst. Rev., 42(6):10--16, 2008.
[11]
A. Gulati, I. Ahmad, and C. A. Waldspurger. PARDA: Proportional Allocation of Resources for Distributed Storage Access. In FAST '09, San Francisco, California, USA, February 2009.
[12]
J. L. Hellerstein, Y. Diao, S. Parekh, and D. M. Tilbury. Feedback Control of Computing Systems. 2004.
[13]
L. Huang, G. Peng, and T.-c. Chiueh. Multi-dimensional storage virtualization. SIGMETRICS Perform. Eval. Rev., 32(1):14--24, 2004.
[14]
W. Jin, J. S. Chase, and J. Kaur. Interposed proportional sharing for a storage service utility. SIGMETRICS Perform. Eval. Rev., 32(1):37--48, 2004.
[15]
E. Kalyvianaki, T. Charalambous, and S. Hand. Self-adaptive and self-configured cpu resource provisioning for virtualized servers using kalman filters. In ICAC '09, pages 117--126, 2009.
[16]
M. Karlsson, C. Karamanolis, and J. Chase. Controllable fair queuing for meeting performance goals. Perform. Eval., 62(1--4):278--294, 2005.
[17]
M. Karlsson, C. Karamanolis, and X. Zhu. Triage: Performance differentiation for storage systems using adaptive control. Trans. Storage, 1(4):457--480, 2005.
[18]
S. Keshav. A control-theoretic approach to flow control. SIGCOMM Comput. Commun. Rev., 21(4):3--15, 1991.
[19]
C. Lu, G. A. Alvarez, and J. Wilkes. Aqueduct: Online data migration with performance guarantees. In FAST '02, page 21, 2002.
[20]
C. R. Lumb, A. Merchant, and G. A. Alvarez. Facade: Virtual storage devices with performance guarantees. In FAST '03, pages 131--144, 2003.
[21]
D. Meister and A. Brinkmann. Multi-level comparison of data deduplication in a backup scenario. In SYSTOR '09, pages 1--12, 2009.
[22]
N. Mi, A. Riska, X. Li, E. Smirni, and E. Riedel. Restrained utilization of idleness for transparent scheduling of background tasks. In SIGMETRICS '09, pages 205--216, 2009.
[23]
N. Mi, A. Riska, Q. Zhang, E. Smirni, and E. Riedel. Efficient management of idleness in storage systems. In ACM Transactions on Storage (TOS), volume 5, pages 1--25, June 2009.
[24]
NEC Corporation. HYDRAstor Grid Storage System, 2008. http://www.hydrastor.com.
[25]
P. Padala, K.-Y. Hou, K. G. Shin, X. Zhu, M. Uysal, Z. Wang, S. Singhal, and A. Merchant. Automated control of multiple virtualized resources. In EuroSys '09, pages 13--26, 2009.
[26]
S.-M. Park and M. Humphrey. Feedback-controlled resource sharing for predictable escience. In SC '08, pages 1--11, Piscataway, NJ, USA, 2008. IEEE Press.
[27]
A. Popescu and S. Ghanbari. A study on performance isolation approaches for consolidated storage. Technical report, May 2008.
[28]
B. Quetier, V. Neri, and F. Cappello. Selecting a virtualization system for grid/p2p large scale emulation. In EXPGRID '06, June 2006.
[29]
C. Ungureanu, A. Aranya, S. Gokhale, S. Rago, B. Atkin, A. Bohra, C. Dubnicki, and G. Calkowski. Hydrafs: A high-throughput file system for the hydrastor content-addressable storage system. In FAST '10, pages 225--239, 2010.
[30]
Y. Wang and A. Merchant. Proportional-share scheduling for distributed storage systems. In FAST '07, pages 4--4, 2007.
[31]
M. Welsh and D. Culler. Adaptive overload control for busy internet servers. In USITS'03, pages 4--4, 2003.
[32]
J. Xu, M. Zhao, J. Fortes, R. Carpenter, and M. Yousif. On the Use of Fuzzy Modeling in Virtualized Data Center Management. In 4th IEEE International Conference on Autonomic Computing, Washington, DC, USA, June 2007.
[33]
J. Zhang, A. Sivasubramaniam, A. Riska, Q. Wang, and E. Riedel. An interposed 2-level i/o scheduling framework for performance virtualization. In SIGMETRICS '05, pages 406--407, 2005.
[34]
X. Zhu, X. Liu, S. Singhal, and M. Arlitt. Resource entitlement control system, January 2010. US patent application no 7644162.
[35]
X. Zhu, M. Uysal, Z. Wang, S. Singhal, A. Merchant, P. Padala, and K. Shin. What does control theory bring to systems research? volume 43, pages 62--69, 2009.

Cited By

View all
  • (2023)Derrick: A Three-layer Balancer for Self-managed Continuous ScalabilityACM Transactions on Storage10.1145/359454319:3(1-34)Online publication date: 28-Apr-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SYSTOR '13: Proceedings of the 6th International Systems and Storage Conference
June 2013
198 pages
ISBN:9781450321167
DOI:10.1145/2485732
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 June 2013

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Conference

SYSTOR '13
Sponsor:
  • INTEL
  • Riverbed
  • Technion
  • SIGOPS
  • EMC<sup>2</sup>
  • AXCIENT
  • USENIX Assoc
  • IBM
  • HP

Acceptance Rates

SYSTOR '13 Paper Acceptance Rate 20 of 49 submissions, 41%;
Overall Acceptance Rate 108 of 323 submissions, 33%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 22 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Derrick: A Three-layer Balancer for Self-managed Continuous ScalabilityACM Transactions on Storage10.1145/359454319:3(1-34)Online publication date: 28-Apr-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media