Ming Chen
Erez Zadok
ABSTRACT
The popular Network File System (NFS) protocol is 30 years old. The latest version, NFSv4, is more than ten years old but has only recently gained stability and acceptance. NFSv4 is vastly different from its predecessors: it offers a stateful server, strong security, scalability/WAN features, and callbacks, among other things. Yet NFSv4's efficacy and ability to meet its stated design goals had not been thoroughly studied until now. This paper compares NFSv4.1's performance with NFSv3 using a wide range of micro- and macro-benchmarks on a testbed configured to exercise the core protocol features. We (1) tested NFSv4's unique features, such as delegations and statefulness; (2) evaluated performance comprehensively with different numbers of threads and clients, and different network latencies and TCP/IP features; (3) found, fixed, and reported several problems in Linux's NFSv4.1 implementation, which helped improve performance by up to 11X; and (4) discovered, analyzed, and explained several counter-intuitive results. Depending on the workload, NFSv4.1 was up to 67\% slower than NFSv3 in a low-latency network, but exceeded NFSv3's performance by up to 2.9X in a high-latency environment. Moreover, NFSv4.1 outperformed NFSv3 by up to 172X when delegations were used.
- A. Bessani and R. Mendes and T. Oliveira and N. Neves and M. Correia and M. Pasin and P. Verissimo. SCFS: A Shared Cloud-backed File System. In USENIX ATC 14, pages 169--180. USENIX, 2014. Google Scholar
Digital Library
- N. Agrawal, V. Prabhakaran, T. Wobber, J. Davis, M. Manasse, and R. Panigraphy. Design tradeoffs for SSD performance. In Proceedings of the USENIX Annual Technical Conference, pages 57--70, Boston, MA, June 2008. Google ScholarDigital Library
- M. Alizadeh, A. Greenberg, D. Maltz, J. Padhye, P. Patel, B. Prabhakar, S. Sengupta, and M. Sridharan. Data Center TCP. In Proceedings of the ACM SIGCOMM 2010 Conference, pages 63--74, 2010.Google ScholarDigital Library
- B. Harrington, A. Charbon, T. Reix, V. Roqueta, J. B. Fields, T. Myklebust, and S. Jayaraman. NFSv4 test project. In Linux Simposium, pages 115--134, 2006.Google Scholar
- A. Batsakis and R. Burns. Cluster delegation: High-performance, fault-tolerant data sharing in NFS. In Proceedings of the 14th IEEE International Symposium on High Performance Distributed Computing. IEEE, July 2005. Google ScholarDigital Library
- Christopher M. Boumenot. The performance of a Linux NFS implementation. Master's thesis, Worcester Polytechnic Institute, May 2002.Google Scholar
- B. Callaghan, B. Pawlowski, and P. Staubach. NFS Version 3 Protocol Specification. Technical Report RFC 1813, Network Working Group, June 1995. Google Scholar
- D. Ellard and M. Seltzer. NFS tricks and benchmarking traps. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track, pages 101--114, San Antonio, TX, June 2003. USENIX Association. Google ScholarDigital Library
- Sorin Faibish. NFSv4.1 and pNFS ready for prime time deployment, 2011.Google Scholar
- John Fastabend and Eric Dumazet. {BUG} ixgbe: only num_online_cpus() of the tx queues are enabled, 2014. http://comments.gmane.org/gmane.linux.network/307532.Google Scholar
- Bruce Fields. NFSv4.1 server implementation. http://goo.gl/vAqR0M.Google Scholar
- A. Gulati, M. Naik, and R. Tewari. Nache: Design and Implementation of a Caching Proxy for NFSv4. In Proceedings of the Fifth USENIX Conference on File and Storage Technologies (FAST '07), pages 199--214, San Jose, CA, February 2007. USENIX Association. Google ScholarDigital Library
- D. Hildebrand and P. Honeyman. Exporting storage systems in a scalable manner with pNFS. In Proceedings of MSST, Monterey, CA, 2005. IEEE. Google ScholarDigital Library
- Eric Kustarz. Using Filebench to evaluate Solaris NFSv4, 2005. NAS conference talk.Google Scholar
- A. W. Leung, S. Pasupathy, G. Goodson, and E. L. Miller. Measurement and analysis of large-scale network file system workloads. In Proceedings of the USENIX Annual Technical Conference, pages 213--226, Boston, MA, June 2008. Google ScholarDigital Library
- C. Lever and P. Honeyman. Linux NFS Client Write Performance. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track, pages 29--40, Monterey, CA, June 2002. USENIX Association. Google ScholarDigital Library
- M. Eshel and R. Haskin and D. Hildebrand and M. Naik and F. Schmuck and R. Tewari. Panache: A Parallel File System Cache for Global File Access. In FAST, pages 155--168. USENIX, 2010. Google ScholarDigital Library
- Ben Martin. Benchmarking NFSv3 vs. NFSv4 file operation performance, 2008. Linux.com.Google Scholar
- R. Martin and D. Culler. NFS Sensitivity to High Performance Networks. In Proceedings of SIGMETRICS. ACM, 1999. Google ScholarDigital Library
- Alex McDonald. The background to NFSv4.1. ;login: The USENIX Magazine, 37(1):28--35, February 2012.Google Scholar
- Paul E. McKenney. Stochastic fairness queueing. In INFOCOM'90, pages 733--740. IEEE, 1990.Google ScholarCross Ref
- Trond Myklebust. File creation speedups for NFSv4.2, 2010.Google Scholar
- P. Radkov, L. Yin, P. Goyal, P. Sarkar, and P. Shenoy. A performance comparison of NFS and iSCSI for IP-networked storage. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST), pages 101--114, San Francisco, CA, March/April 2004. USENIX Association. Google ScholarDigital Library
- Scott Rixner. Network virtualization: Breaking the performance barrier. Queue, 6(1):37:36--37:ff, Jan 2008. Google ScholarDigital Library
- S. Shepler and M. Eisler and D. Noveck. NFS Version 4 Minor Version 1 Protocol. Technical Report RFC 5661, Network Working Group, January 2010.Google Scholar
- SPEC. SPECsfs2008. www.spec.org/sfs2008, 2008.Google Scholar
- V. Tarasov, S. Bhanage, E. Zadok, and M. Seltzer. Benchmarking File System Benchmarking: It *IS* Rocket Science. In Proceedings of HotOS XIII:The 13th USENIX Workshop on Hot Topics in Operating Systems, Napa, CA, May 2011. Google ScholarDigital Library
- Vasily Tarasov, Dean Hildebrand, Geoff Kuenning, and Erez Zadok. Virtual machine workloads: The case for new benchmarks for NAS. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST), San Jose, CA, February 2013. USENIX Association. Google ScholarDigital Library
- A. W. Wilson. Operation and implementation of random variables in Filebench.Google Scholar
- M. Wittle and B. E. Keith. LADDIS: The next generation in NFS file server benchmarking. In Proceedings of the Summer USENIX Technical Conference, pages 111--128, Cincinnati, OH, June 1993. USENIX Association. Google ScholarDigital Library
- N. Yezhkova, L. Conner, R. Villars, and B. Woo. Worldwide enterprise storage systems 2010--2014 forecast: recovery, efficiency, and digitization shaping customer requirements for storage systems. IDC, May 2010. IDC#223234.Google Scholar
Index Terms
- Newer Is Sometimes Better: An Evaluation of NFSv4.1
Recommendations
Newer Is Sometimes Better: An Evaluation of NFSv4.1
Performance evaluation reviewThe popular Network File System (NFS) protocol is 30 years old. The latest version, NFSv4, is more than ten years old but has only recently gained stability and acceptance. NFSv4 is vastly different from its predecessors: it offers a stateful server, ...
Implementation of a stackable file system for real-time network backup
We propose a backup system based on a stackable mirroring file system, general-purpose mirroring file system (GMFS). This file system mirrors data in real-time on the file system layer. It uses the typical network file system (NFS) and backs up data to ...
Comments