skip to main content
article

Versatility and Unix semantics in namespace unification

Published: 01 February 2006 Publication History

Abstract

Administrators often prefer to keep related sets of files in different locations or media, as it is easier to maintain them separately. Users, however, prefer to see all files in one location for convenience. One solution that accommodates both needs is virtual namespace unification---providing a merged view of several directories without physically merging them. For example, namespace unification can merge the contents of several CD-ROM images without unpacking them, merge binary directories from different packages, merge views from several file servers, and more. Namespace unification can also enable snapshotting by marking some data sources read-only and then utilizing copy-on-write for the read-only sources. For example, an OS image may be contained on a read-only CD-ROM image---and the user's configuration, data, and programs could be stored in a separate read-write directory. With copy-on-write unification, the user need not be concerned about the two disparate file systems.It is difficult to maintain Unix semantics while offering a versatile namespace unification system. Past efforts to provide such unification often compromised on the set of features provided or Unix compatibility---resulting in an incomplete solution that users could not use.We designed and implemented a versatile namespace unification system called Unionfs. Unionfs maintains Unix semantics while offering advanced namespace unification features: dynamic insertion and removal of namespaces at any point in the merged view, mixing read-only and read-write components, efficient in-kernel duplicate elimination, NFS interoperability, and more. Since releasing our Linux implementation, it has been used by thousands of users and over a dozen Linux distributions, which helped us discover and solve many practical problems.

References

[1]
AT&T Bell Laboratories. 1995. Plan 9---Programmer's Manual. AT&T Bell Laboratories.]]
[2]
Ellard, D. and Seltzer, M. 2003. NFS tricks and benchmarking traps. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track (San Antonio, Tex.), 101--114.]]
[3]
Flouris, M. D. and Bilas, A. 2004. Clotho: Transparent data versioning at the block I/O level. In Proceedings of the 12th NASA Goddard, 21st IEEE Conference on Mass Storage Systems and Technologies (MSST 2004). College Park, Md. 315--328.]]
[4]
Fraser, T., Badger, L., and Feldman, M. 1999. Hardening COTS software with generic software wrappers. In Proceedings of the 1999 IEEE Symposium on Security and Privacy, 2--16.]]
[5]
Guy, R. G., Heidemann, J. S., Mak, W., Page Jr., T. W., Popek, G. J., and Rothmeier, D. 1990. Implementation of the Ficus replicated file system. In Proceedings of the Summer USENIX Technical Conference, 63--71.]]
[6]
Heidemann, J. S. and Popek, G. J. 1994. File system development with stackable layers. ACM Trans. Comput. Syst. 12, 1 (Feb.), 58--89.]]
[7]
Hendricks, D. 1990. A filesystem for software development. In Proceedings of the USENIX Summer Conference (Anaheim, Calif.), 333--340.]]
[8]
Hitz, D., Lau, J., and Malcolm, M. 1994. File system design for an NFS file server appliance. In Proceedings of the USENIX Winter Technical Conference (San Francisco, Calif.), 235--245.]]
[9]
Joukov, N., Wright, C. P., and Zadok, E. 2004. FSprof: An in-kernel file system operations profiler. Tech. Rep. FSL-04-06, Computer Science Department, Stony Brook University. www.fsl.cs.sunysb.edu/docs/aggregate_stats-tr/aggregate_stats.pdf.]]
[10]
Kamp, P. H. and Watson, R. N. M. 2000. Jails: Confining the omnipotent root. In Proceedings of the 2nd International System Administration and Networking Conference (SANE2000). Maastricht, The Netherlands.]]
[11]
Katcher, J. 1997. PostMark: A new filesystem benchmark. Tech. Rep. TR3022, Network Appliance. www.netapp.com/tech_library/3022.html.]]
[12]
Korn, D. G. and Krell, E. 1990. A new dimension for the Unix file system. Softw. Prac. Expe., 19--34.]]
[13]
McKusick, M. K. and Ganger, G. R. 1999. Soft updates: A technique for eliminating most synchronous writes in the fast filesystem. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track (Monterey, Calif.), 1--18.]]
[14]
OpenBSD. 2005. OpenSSH. www.openssh.org.]]
[15]
Pendry, J. S. and McKusick, M. K. 1995. Union mounts in 4.4BSD-Lite. In Proceedings of the USENIX Technical Conference on UNIX and Advanced Computing Systems, 25--33.]]
[16]
Pendry, J. S., Williams, N., and Zadok, E. 2003. Am-utils User Manual, 6.1b3 ed. www.am-utils.org.]]
[17]
Peterson, Z. and Burns, R. 2005. Ext3cow: A time-shifting file system for regulatory compliance. Trans. Storage 1, 2, 190--212.]]
[18]
Roselli, D., Lorch, J. R., and Anderson, T. E. 2000. A comparison of file system workloads. In Proceedings of the Annual USENIX Technical Conference (San Diego, Calif.), 41--54.]]
[19]
Rosenthal, D. S. H. 1990. Evolving the Vnode interface. In Proceedings of the Summer USENIX Technical Conference, 107--18.]]
[20]
Spadavecchia, J. and Zadok, E. 2002. Enhancing NFS cross-administrative domain access. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track (Monterey, Calif.), 181--194.]]
[21]
Wright, C. P., Martino, M., and Zadok, E. 2003. NCryptfs: A secure and convenient cryptographic file system. In Proceedings of the Annual USENIX Technical Conference (San Antonio, Tex.), 197--210.]]
[22]
Zadok, E. and Nieh, J. 2000. FiST: A language for stackable file systems. In Proceedings of the Annual USENIX Technical Conference (San Diego, Calif.), 55--70.]]

Cited By

View all
  • (2022)Self-deployed execution environment for high performance computing面向高性能计算的自部署运行环境Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.210001623:6(845-857)Online publication date: 4-Mar-2022
  • (2022)Container-aware I/O stack: bridging the gap between container storage drivers and solid state devicesProceedings of the 18th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments10.1145/3516807.3516818(18-30)Online publication date: 25-Feb-2022
  • (2022)FUSE based file system for efficient storage and retrieval of fragmented multimedia filesJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2022.08.01834:10(8380-8389)Online publication date: Nov-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage
ACM Transactions on Storage  Volume 2, Issue 1
February 2006
105 pages
ISSN:1553-3077
EISSN:1553-3093
DOI:10.1145/1138041
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 February 2006
Published in TOS Volume 2, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Namespace management
  2. directory merging
  3. snapshotting
  4. stackable file systems
  5. unification

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)13
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Self-deployed execution environment for high performance computing面向高性能计算的自部署运行环境Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.210001623:6(845-857)Online publication date: 4-Mar-2022
  • (2022)Container-aware I/O stack: bridging the gap between container storage drivers and solid state devicesProceedings of the 18th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments10.1145/3516807.3516818(18-30)Online publication date: 25-Feb-2022
  • (2022)FUSE based file system for efficient storage and retrieval of fragmented multimedia filesJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2022.08.01834:10(8380-8389)Online publication date: Nov-2022
  • (2021)DeltaFSProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3458817.3476148(1-15)Online publication date: 14-Nov-2021
  • (2021)The RESTless cloudProceedings of the Workshop on Hot Topics in Operating Systems10.1145/3458336.3465280(49-57)Online publication date: 1-Jun-2021
  • (2021)Prevention of a DoS Attack with Copy-on-write in the Overlay Filesystem2021 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech)10.1109/DASC-PICom-CBDCom-CyberSciTech52372.2021.00026(76-83)Online publication date: Oct-2021
  • (2020)A NAS Integrated File System for On-site IoT Data Storage2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)10.1109/PerComWorkshops48775.2020.9156143(1-7)Online publication date: Mar-2020
  • (2020) Load-aware Adaptive Cache Management Scheme for Enterprise-level Stackable Cryptographic File System * 2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC/SmartCity/DSS)10.1109/HPCC-SmartCity-DSS50907.2020.00006(35-43)Online publication date: Dec-2020
  • (2016)SeMiNASProceedings of the 9th ACM International on Systems and Storage Conference10.1145/2928275.2928282(1-13)Online publication date: 6-Jun-2016
  • (2016)MultiLanesACM Transactions on Storage10.1145/280115512:3(1-31)Online publication date: 16-Jun-2016
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media