research-article

CA-NFS: A congestion-aware network file system

Authors:

Alexandros Batsakis,

Arkady Kanevsky,

Thomas TalpeyAuthors Info & Claims

ACM Transactions on Storage (TOS), Volume 5, Issue 4

Article No.: 15, Pages 1 - 24

https://doi.org/10.1145/1629080.1629085

Published: 14 December 2009 Publication History

Abstract

We develop a holistic framework for adaptively scheduling asynchronous requests in distributed file systems. The system is holistic in that it manages all resources, including network bandwidth, server I/O, server CPU, and client and server memory utilization. It accelerates, defers, or cancels asynchronous requests in order to improve application-perceived performance directly. We employ congestion pricing via online auctions to coordinate the use of system resources by the file system clients so that they can detect shortages and adapt their resource usage. We implement our modifications in the Congestion-Aware Network File System (CA-NFS), an extension to the ubiquitous network file system (NFS). Our experimental result shows that CA-NFS results in a 20% improvement in execution times when compared with NFS for a variety of workloads.

References

[1]

Alonso, M. and Santonja, V. 1999. A new destage algorithm for disk cache: DOME. In Proceedings of the EUROMICRO Conference.

[2]

Amir, Y., Awerbuch, B., Danilov, C., and Stanton, J. 2005. A cost-benefit flow control for reliable multicast and unicast in overlay networks. Trans. Netw.

Digital Library

[3]

Anderson, E., Spence, S., Swaminathan, R., Kallahalla, M., and Wang, Q. 2005. Quickly finding near-optimal storage designs. Trans. Comput. Syst.

Digital Library

[4]

Awerbuch, B., Azar, Y., Plotkin, S. A., and Waarts, O. 1994. Competitive routing of virtual circuits with unknown duration. In Proceedings of the Symposium on Discrete Algorithms. ACM, New York, NY.

Digital Library

[5]

Baker, M. G., Hartman, J. H., Kupfer, M. D., Shirriff, K. W., and Ousterhout, J. K. 1991. Measurements of a distributed file system. In Proceedings of the Symposium on Operating Systems Principles. ACM, New York, NY.

Digital Library

[6]

Batsakis, A., Burns, R., Kanevsky, A., Lentini, J., and Talpey, T. 2008. AWOL: An adaptive write optimizations in layer. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA.

Digital Library

[7]

Capps, D. 2009. The IOzone Benchmark. http://www.iozone.com.

[8]

Carson, S. D. and Setia, S. 1992. Analysis of the periodic update write policy for disk cache. Trans. Softw. Engin. 18.

Digital Library

[9]

Chambliss, D. D., Alvarez, G. A., Pandey, P., Jadav, D., Xu, J., Menon, R., and Lee, T. P. 2003. Performance virtualization for large-scale storage systems. In Proceedings of the Symposium on Reliable Distributed Systems. IEEE.

[10]

Chun, B. N. and Culler, D. E. 2002. User-centric performance analysis of market-based cluster batch schedulers. In Proceedings of the IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02). ACM, New York, NY.

Digital Library

[11]

Clearwater, S. 1996. Market-Based Control: A Paradigm for Distributed Resource Allocation World Scientific, Kirkkonummi, Finland.

Digital Library

[12]

Druschel, P. and Banga, G. 1996. Lazy receiver processing (LRP): A network subsystem architecture for server systems. In Proceedings of the Symposium on Operating Systems Design and Implementation. USENIX, Berkeley, CA.

Digital Library

[13]

Fengguang, W. 2007. Adaptive read-ahead in the Linux kernel. http://lwn.net/Articles/155097/.

[14]

Floyd, S. 1994. TCP and explicit congestion notification. Comput. Comm. Rev. 24, 5, 10--23.

Digital Library

[15]

Floyd, S. and Jacobson, V. 1993. Random early detection gateways for congestion avoidance. Trans. Netw. 1, 4, 397--413.

Digital Library

[16]

Golding, R., Bosch, P., Staelin, C., Sullivan, T., and Wilkes, J. 1995. Idleness is not sloth. In Proceedings of the USENIX Annual Technical Conference. USENIX, Berkeley, CA.

Digital Library

[17]

Goyal, P., Jadav, D., Modha, D. S., and Tewari, R. 2003. CacheCOW: QoS for storage system caches. In Proceedings of the International Workshop on Quality of Service (IWQoS'03). ACM, New York, NY.

Digital Library

[18]

Jacobson, V. 1988. Congestion avoidance and control. In Proceedings of the ACM SIGCOMM Communications Festival. ACM, New York, NY.

Digital Library

[19]

Jones, M. B., Rou, D., and Rou, M. 1997. Cpu reservations and time constraints: Efficient, predictable scheduling of independent activities. In Proceedings of the Symposium of Operating Systems and Principles. ACM, New York, NY.

Digital Library

[20]

Kaldewey, T., Wong, T., Golding, R., Povzner, A., Brandt, S. A., and Maltzahn, C. 2008. Virtualizing disk performance. In Proceedings of the Real-Time and Embedded Technology and Applications Symposium. IEEE, Los Alamitos, CA.

Digital Library

[21]

Karlsson, M., Karamanolis, C., and Zhu, X. 2004. Triage: Performance isolation and differentiation for storage systems. In Proceedings of the International Workshop on Quality of Service.

[22]

Kelly, F., Maulloo, A., and Tan, D. 1998. Rate control in communication networks: Shadow prices, proportional fairness and stability. J. Oper. Resear. Soc. 49.

[23]

Ki, A. and Knowles, A. E. 1997. Adaptive data prefetching using cache information. In Proceedings of the International Conference on Supercomputing. IEEE, New York, NY.

Digital Library

[24]

Kustarz, E., Shepler, S., and A.Wilson. 2008. The new and improved filebench file system benchmarking framework. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA.

[25]

Lai, K., Rasmusson, L., Adar, E., Zhang, L., and Huberman, B. A. 2005. Tycoon: An implementation of a distributed, market-based resource allocation system. Multiagent Grid Syst. 1, 3, 169--182.

Digital Library

[26]

Levy, L., Blumrosen, L., and Nisan, N. 2001. On line markets for distributed object services: The majic system. In Proceedings of the USENIX Symposium on Internet Technologies and Systems (USITS'01). USENIX, Berkeley, CA.

Digital Library

[27]

Li, X., Aboulnaga, A., Salem, K., Sachendina, A., and Gao, S. 2005. Second-tier cache management using write hints. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA.

Digital Library

[28]

Lu, Y., Abdelzaher, T., Lu, C., and Tao, G. 2002. An adaptive control framework for qos guarantees and its application to differentiated caching services. In Proceedings of the International Workshop on Quality of Service. IEEE, Los Alamitos, CA.

[29]

Lumb, C., Merchant, A., and Alvarez, G. 2003. Facade: Virtual storage devices with performance guarantees. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA.

Digital Library

[30]

Massoulie, L. and Roberts, J. 1999. Bandwidth sharing: Objectives and algorithms. In Proceedings of the Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM). IEEE, Los Alamitos, CA.

[31]

Mogul, J. 1994. A better update policy. In USENIX Summer Technical Conference. USENIX, Berkeley, CA, 1--10.

Digital Library

[32]

Nam, Y. J. and Park, C. 2002. An adaptive high-low water mark destage algorithm for cached RAID5. In Proceedings of the Pacific Rim Internataional Symposium on Dependable Computing. IEEE, Los Alamitos, CA.

Digital Library

[33]

Nelson, M. N., Welch, B. B., and Ousterhout, J. K. 1988. Caching in the sprite network file system. Trans. Comput. Syst. 6, 1, 1--20.

Digital Library

[34]

Patterson, R. H., Gibson, G. A., Ginting, E., Stodolsky, D., and Zelenka, J. 1995. Informed prefetching and caching. In Proceedings of the ACM Symposium on Operating Systems Principles. ACM, New York, NY.

Digital Library

[35]

Povzner, A., Kaldewey, T., Brandt, S., Golding, R., Wong, T. M., and Maltzahn, C. 2008. Efficient guaranteed disk request scheduling with fahrrad. SIGOPS Oper. Syst. Rev. 42, 4, 13--25.

Digital Library

[36]

Revel, D., McNamee, D., Steere, D., and Walpole, J. 1997. Adaptive prefetching for device independent file I/O. Tech. rep. CSE-97-005, School of Science and Engineering, Oregon Graduate Institute.

Digital Library

[37]

Riska, A., Riedel, E., and Iren, S. 2004. Adaptive disk scheduling for overload management. In Proceedings of the International Conference on the Quantitative Evaluation of Systems. IEEE, Los Alamitos, CA.

Digital Library

[38]

Stoica, I., Abdel-Wahab, H., and Pothen, A. 1995. A Microeconomic Scheduler for Parallel Computers. In Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing. ACM, New York, NY.

Digital Library

[39]

Strunk, J. D., Thereseka, E., Faloutsos, C., and Ganger, G. R. 2008. Using utility to provision storage systems. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA.

Digital Library

[40]

Uttamchandani, S., Yin, L., Alvarez, G. A., Palmer, J., and Agha, G. 2005. Chameleon: A self-evolving, fully-adaptive resource arbitrator for storage systems. In Proceedings of the USENIX Annual Technical Conference. USENIX, Berkeley, CA.

Digital Library

[41]

Varma, A. and Jacobsen, Q. 1998. Destage algorithms for disk arrays with nonvolatile caches. Trans. Comput. 47, 2, 228--235.

Digital Library

[42]

Welsh, M., Culler, D., and Brewer, E. 2001. Seda: An architecture for well-conditioned scalable internet services. In Proceedings of the Symposium of Operating Systems Principles. ACM, New York, NY.

Digital Library

[43]

Zimmermann, Z. and Killat, U. 2002. Resource marking and fair rate allocation. In Proceedings of the International Conference on Communications. IEEE, Los Alamitos, CA, 1310--1314.

Cited By

Roy RPatel TTiwari D(2021)Characterizing and Mitigating the I/O Scalability Challenges for Serverless Applications2021 IEEE International Symposium on Workload Characterization (IISWC)10.1109/IISWC53511.2021.00018(74-86)Online publication date: Nov-2021
https://doi.org/10.1109/IISWC53511.2021.00018
Guha SLi YZhang Q(2019)Distributed Partial ClusteringACM Transactions on Parallel Computing10.1145/33228086:3(1-20)Online publication date: 15-Oct-2019
https://dl.acm.org/doi/10.1145/3322808
Deutsch ALi YVianu V(2019)Verification of Hierarchical Artifact SystemsACM Transactions on Database Systems10.1145/332148744:3(1-68)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3321487
Show More Cited By

Index Terms

CA-NFS: A congestion-aware network file system
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Memory management
        Distributed memory
    2. Software system structures
      1. Distributed systems organizing principles

Recommendations

Implementation of a stackable file system for real-time network backup

We propose a backup system based on a stackable mirroring file system, general-purpose mirroring file system (GMFS). This file system mirrors data in real-time on the file system layer. It uses the typical network file system (NFS) and backs up data to ...
Using Eager Strategies to Improve NFS I/O Performance
NAS '11: Proceedings of the 2011 IEEE Sixth International Conference on Networking, Architecture, and Storage

Typical NFS clients write in a lazy fashion: they leave dirty pages in the page cache and defer writing to the server until later. This reduces network traffic when applications repeatedly modify the same set of pages. However, this approach can lead to ...
RFC 6688: Parallel NFS (pNFS) Block Disk Protection

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage

ACM Transactions on Storage Volume 5, Issue 4

December 2009

155 pages

ISSN:1553-3077

EISSN:1553-3093

DOI:10.1145/1629080

Issue’s Table of Contents

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 December 2009

Accepted: 01 August 2009

Received: 01 August 2009

Published in TOS Volume 5, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
528
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Roy RPatel TTiwari D(2021)Characterizing and Mitigating the I/O Scalability Challenges for Serverless Applications2021 IEEE International Symposium on Workload Characterization (IISWC)10.1109/IISWC53511.2021.00018(74-86)Online publication date: Nov-2021
https://doi.org/10.1109/IISWC53511.2021.00018
Guha SLi YZhang Q(2019)Distributed Partial ClusteringACM Transactions on Parallel Computing10.1145/33228086:3(1-20)Online publication date: 15-Oct-2019
https://dl.acm.org/doi/10.1145/3322808
Deutsch ALi YVianu V(2019)Verification of Hierarchical Artifact SystemsACM Transactions on Database Systems10.1145/332148744:3(1-68)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3321487
Ellis SKlinge TLathrop JLutz JLutz RMiner APotter H(2019)Runtime Fault Detection in Programmed Molecular SystemsACM Transactions on Software Engineering and Methodology10.1145/329574028:2(1-20)Online publication date: 13-Mar-2019
https://dl.acm.org/doi/10.1145/3295740
Gołębiewski ZMagner ASzpankowski W(2018)Entropy and Optimal Compression of Some General Plane TreesACM Transactions on Algorithms10.1145/327544415:1(1-23)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1145/3275444
Zhou DFang VXie TPan WKesavan RLin TPatel N(2018)Empirical Evaluation and Enhancement of Enterprise Storage System Request SchedulingACM Transactions on Storage10.1145/319374114:2(1-27)Online publication date: 27-Apr-2018
https://dl.acm.org/doi/10.1145/3193741
Chen MHildebrand DNelson HSaluja JSubramony AZadok EKuenning GWaldspurger C(2017)vNFSProceedings of the 15th Usenix Conference on File and Storage Technologies10.5555/3129633.3129661(301-314)Online publication date: 27-Feb-2017
https://dl.acm.org/doi/10.5555/3129633.3129661
Piccolboni LMantovani PGuglielmo GCarloni L(2017)COSMOSACM Transactions on Embedded Computing Systems10.1145/312656616:5s(1-22)Online publication date: 27-Sep-2017
https://dl.acm.org/doi/10.1145/3126566
Rouhani BMirhoseini AKoushanfar F(2017)RISEACM Transactions on Embedded Computing Systems10.1145/312654916:5s(1-18)Online publication date: 27-Sep-2017
https://dl.acm.org/doi/10.1145/3126549
Lesi VJovanov IPajic M(2017)Security-Aware Scheduling of Embedded Control TasksACM Transactions on Embedded Computing Systems10.1145/312651816:5s(1-21)Online publication date: 27-Sep-2017
https://dl.acm.org/doi/10.1145/3126518
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents