skip to main content
research-article

Recon: Verifying file system consistency at runtime

Published: 06 December 2012 Publication History

Abstract

File system bugs that corrupt metadata on disk are insidious. Existing reliability methods, such as checksums, redundancy, or transactional updates, merely ensure that the corruption is reliably preserved. Typical workarounds, based on using backups or repairing the file system, are painfully slow. Worse, the recovery may result in further corruption.
We present Recon, a system that protects file system metadata from buggy file system operations. Our approach leverages file systems that provide crash consistency using transactional updates. We define declarative statements called consistency invariants for a file system. These invariants must be satisfied by each transaction being committed to disk to preserve file system integrity. Recon checks these invariants at commit, thereby minimizing the damage caused by buggy file systems.
The major challenges to this approach are specifying invariants and interpreting file system behavior correctly without relying on the file system code. Recon provides a framework for file-system specific metadata interpretation and invariant checking. We show the feasibility of interpreting metadata and writing consistency invariants for the Linux ext3 file system using this framework. Recon can detect random as well as targeted file-system corruption at runtime as effectively as the offline e2fsck file-system checker, with low overhead.

References

[1]
Arnold, J. and Kaashoek, M. F. 2009. Ksplice: automatic rebootless kernel updates. In Proceedings of the ACM SIGOPS European Conference on Computer Systems (EuroSys'09). 187--198.
[2]
Bairavasundaram, L. N., Arpaci-Dusseau, A. C., Arpaci-Dusseau, R. H., Goodson, G. R., and Schroeder, B. 2008. An analysis of data corruption in the storage stack. ACM Trans. Storage 4, 3, 1--28.
[3]
Bairavasundaram, L. N., Sundararaman, S., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2009. Tolerating file-system mistakes with EnvyFS. In Proceedings of the USENIX Technical Conference. 87--100.
[4]
Bonwick, J. and Moore, B. 2004. ZFS: the last word in file systems. http://opensolaris.org/os/community/zfs/docs/zfs_last.pdf.
[5]
Btrfs. 2012. Wikipedia page. http://btrfs.wiki.kernel.org.
[6]
Chen, F. and Roşu, G. 2007. Mop: an efficient and generic runtime verification framework. In Proceedings of the ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA'07). 569--588.
[7]
Custer, H. 1994. Inside the Windows NT File System. Microsoft Press.
[8]
Danial, A. 2012. CLOC: count lines of code. http://cloc.sourceforge.net/.
[9]
Demsky, B. and Rinard, M. C. 2006. Goal-directed reasoning for specification-based data structure repair. IEEE Trans. Softw. Engin. 32, 12, 931--951.
[10]
Ganger, G. R., McKusick, M. K., Soules, C. A. N., and Patt, Y. N. 2000. Soft updates: a solution to the metadata update problem in file systems. ACM Trans. Comput. Syst. 18, 2, 127--153.
[11]
Gunawi, H. S., Prabhakaran, V., Krishnan, S., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2007. Improving file system reliability with I/O shepherding. In Proceedings of the Symposium on Operating Systems Principles (SOSP'07). 293--306.
[12]
Gunawi, H. S., Rajimwale, A., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2008. SQCK: a declarative file system checker. In Proceedings of the Operating Systems Design and Implementation (OSDI'08). 131--146.
[13]
Hagmann, R. 1987. Reimplementing the Cedar file system using logging and group commit. In Proceedings of the Symposium on Operating Systems Principles (SOSP'87). 155--162.
[14]
Henson, V., van de Ven, A., Gud, A., and Brown, Z. 2006. Chunkfs: using divide-and-conquer to improve file system reliability and repair. In Proceedings of the Workshop on Hot Topics in System Dependability (HotDep'06).
[15]
Hitz, D., Lau, J., and Malcolm, M. 1994. File system design for an NFS file server appliance. In Proceedings of the USENIX Technical Conference.
[16]
Iptables. 2012. Wikipedia page. http://en.wikipedia.org/wiki/Iptables.
[17]
Kaashoek, F. M., Engler, D. R., Ganger, G. R., Briceno, H. M., Hunt, R., Mazikres, D., Pinckney, T., Grimm, R., Jannotti, J., and Mackenzie, K. 1997. Application Performance and Flexibility on Exokernel Systems. In Proceedings of the Symposium on Operating Systems Principles (SOSP'97). 52--65.
[18]
Meyer, D. T. and Bolosky, W. J. 2010. A study of practical deduplication. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST'10). 1--14.
[19]
Perkins, J. H., Kim, S., Larsen, S., Amarasinghe, S. P., Bachrach, J., Carbin, M., et al. 2009. Automatically patching errors in deployed software. In Proceedings of the Symposium on Operating Systems Principles (SOSP'09). 87--102.
[20]
Prabhakaran, V., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2005a. Model-based failure analysis of journaling file systems. In Proceedings of the IEEE Dependable Systems and Networks (DSN'05). 802--811.
[21]
Prabhakaran, V., Bairavasundaram, L. N., Agrawal, N., Gunawi, H. S., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2005b. Iron file systems. In Proceedings of the Symposium on Operating Systems Principles (SOSP'05). 206--220.
[22]
Rubio-González, Cindy, Gunawi, S., H., Liblit, B., Arpaci-Dusseau, H., R., Arpaci-Dusseau, and C., A. 2009. Error propagation analysis for file systems. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'09). 270--280.
[23]
Sivathanu, G., Sundararaman, S., and Zadok, E. 2006. Type-safe disks. In Proceedings of the Operating Systems Design and Implementation (OSDI'06). 15--28.
[24]
Sivathanu, M., Prabhakaran, V., Popovici, F. I., Denehy, T. E., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2003. Semantically-smart disk systems. In USENIX Conference on File and Storage Technologies (FAST'03). 73--88.
[25]
Sokolsky, O., Sammapun, U., Lee, I., and Kim, J. 2006. Run-time checking of dynamic properties. Electron. Notes Theor. Comput. Sci. 144, 91--108.
[26]
Sundararaman, S., Subramanian, S., Rajimwale, A., Arpaci-dusseau, A. C., Arpaci-dusseau, R. H., and Swift, M. M. 2010. Membrane: Operating system support for restartable file systems. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST'10). 281--294.
[27]
Sweeney, A., Doucette, D., Hu, W., Anderson, C., Nishimoto, M., and Peck, G. 1996. Scalability in the XFS file system. In Proceedings of the USENIX Technical Conference. 1--14.
[28]
Tweedie, S. C. 1998. Journalling the ext2fs filesystem. In Proceedings of the 4th Annual Linux Expo.
[29]
Yang, J., Sar, C., and Engler, D. 2006a. EXPLODE: a lightweight, general system for finding serious storage system errors. In Proceedings of the Operating Systems Design and Implementation (OSDI'06). 131--146.
[30]
Yang, J., Sar, C., Twohey, P., Cadar, C., and Engler, D. 2006b. Automatically generating malicious disks using symbolic execution. In Proceedings of the IEEE Symposium on Security and Privacy. 243--257.
[31]
Yang, J., Twohey, P., Engler, D., and Musuvathi, M. 2006c. Using model checking to find serious file system errors. ACM Trans. Comput. Systems 24, 4, 393--423.
[32]
Zhang, Y., Rajimwale, A., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2010. End-to-end data integrity for file systems: a ZFS case study. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST'10). 29--42.

Cited By

View all
  • (2024)Shadow Filesystems: Recovering from Filesystem Runtime Errors via Robust Alternative ExecutionProceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems10.1145/3655038.3665942(15-22)Online publication date: 8-Jul-2024
  • (2021)Enforcing situation-aware access control to build malware-resilient file systemsFuture Generation Computer Systems10.1016/j.future.2020.09.035115:C(568-582)Online publication date: 1-Feb-2021
  • (2020)Data Storage Security in the Cloud Environment Using the Honey Pot SystemInternational Journal of e-Collaboration10.4018/IJeC.202010010116:4(1-14)Online publication date: 1-Oct-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage
ACM Transactions on Storage  Volume 8, Issue 4
November 2012
82 pages
ISSN:1553-3077
EISSN:1553-3093
DOI:10.1145/2385603
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2012
Accepted: 01 October 2012
Received: 01 October 2012
Published in TOS Volume 8, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Metadata consistency
  2. file system checker
  3. runtime verification

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)27
  • Downloads (Last 6 weeks)6
Reflects downloads up to 18 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Shadow Filesystems: Recovering from Filesystem Runtime Errors via Robust Alternative ExecutionProceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems10.1145/3655038.3665942(15-22)Online publication date: 8-Jul-2024
  • (2021)Enforcing situation-aware access control to build malware-resilient file systemsFuture Generation Computer Systems10.1016/j.future.2020.09.035115:C(568-582)Online publication date: 1-Feb-2021
  • (2020)Data Storage Security in the Cloud Environment Using the Honey Pot SystemInternational Journal of e-Collaboration10.4018/IJeC.202010010116:4(1-14)Online publication date: 1-Oct-2020
  • (2020)A New Hybrid Document Clustering for PRF-Based Automatic Query Expansion Approach for Effective IRInternational Journal of e-Collaboration10.4018/IJeC.202007010516:3(73-95)Online publication date: 1-Jul-2020
  • (2020)Crowdsourcing-Enabled Crisis Collaborative Decision MakingInternational Journal of e-Collaboration10.4018/IJeC.202007010416:3(49-72)Online publication date: 1-Jul-2020
  • (2020)A Novel Data Consistence Model Based on Virtual Peers in Peer-to-Peer SystemsInternational Journal of e-Collaboration10.4018/IJeC.202007010116:3(1-16)Online publication date: 1-Jul-2020
  • (2020)Permissioned Blockchain Model for End-to-End Trackability in Supply Chain ManagementInternational Journal of e-Collaboration10.4018/IJeC.202001010416:1(45-58)Online publication date: 1-Jan-2020
  • (2020)SpiffyACM Transactions on Storage10.1145/338636816:3(1-39)Online publication date: 4-Aug-2020
  • (2020)Cross-Failure Bug Detection in Persistent Memory ProgramsProceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3373376.3378452(1187-1202)Online publication date: 9-Mar-2020
  • (2019)File Systems are Hard to Test — Learning from XfstestsIEICE Transactions on Information and Systems10.1587/transinf.2018EDP7006E102.D:2(269-279)Online publication date: 1-Feb-2019
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media