research-article

Fine-grained user-space security through virtualization

Authors:

Thomas R. GrossAuthors Info & Claims

VEE '11: Proceedings of the 7th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Pages 157 - 168

https://doi.org/10.1145/1952682.1952703

Published: 09 March 2011 Publication History

Abstract

This paper presents an approach to the safe execution of applications based on software-based fault isolation and policy-based system call authorization. A running application is encapsulated in an additional layer of protection using dynamic binary translation in user-space. This virtualization layer dynamically recompiles the machine code and adds multiple dynamic security guards that verify the running code to protect and contain the application.

The binary translation system redirects all system calls to a policy-based system call authorization framework. This interposition framework validates every system call based on the given arguments and the location of the system call. Depending on the user-loadable policy and an extensible handler mechanism the framework decides whether a system call is allowed, rejected, or redirect to a specific user-space handler in the virtualization layer.

This paper offers an in-depth analysis of the different security guarantees and a performance analysis of libdetox, a prototype of the full protection platform. The combination of software-based fault isolation and policy-based system call authorization imposes only low overhead and is therefore an attractive option to encapsulate and sandbox applications to improve host security.

References

[1]

Acharya, A., and Raje, M. MAPbox: using parameterized behavior classes to confine untrusted applications. In SSYM'00: Proceedings of the 9th conference on USENIX Security Symposium (2000).

Digital Library

[2]

Alexandrov, A., Kmiec, P., and Schauser, K. Consh: Confined execution environment for internet computations, 1999.

[3]

Baratloo, A., Singh, N., and Tsai, T. Transparent run-time defense against stack smashing attacks. In ATEC '00: Proceedings of the annual conference on USENIX Annual Technical Conference (2000).

Digital Library

[4]

Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., and Warfield, A. Xen and the art of virtualization. In SOSP '03 (New York, NY, USA, 2003), pp. 164--177.

Digital Library

[5]

Bauer, M. Paranoid penguin: an introduction to novell apparmor. Linux J. 2006, 148 (2006), 13.

Digital Library

[6]

Bellard, F. QEMU, a fast and portable dynamic translator. In ATEC '05 (Berkeley, CA, USA, 2005), pp. 41--41.

Digital Library

[7]

Bhatkar, E., Duvarney, D. C., and Sekar, R. Address obfuscation: an efficient approach to combat a broad range of memory error exploits. In Proceedings of the 12th USENIX Security Symposium (2003), pp. 105--120.

Digital Library

[8]

Bhatkar, S., Bhatkar, E., Sekar, R., and Duvarney, D. C. Efficient techniques for comprehensive protection from memory error exploits. In Proceedings of the 14th USENIX Security Symposium (2005).

Digital Library

[9]

Bruening, D., Duesterwald, E., and Amarasinghe, S. Design and implementation of a dynamic optimization framework for Windows. In ACM Workshop Feedback-directed Dyn. Opt. (FDDO-4) (2001).

[10]

Bruening, D., Garnett, T., and Amarasinghe, S. An infrastructure for adaptive dynamic optimization. In CGO '03 (Washington, DC, USA, 2003), pp. 265--275.

Digital Library

[11]

Bugnion, E. Dynamic binary translator with a system and method for updating and maintaining coherency of a translation cache. US Patent 6704925, March 2004.

[12]

Cowan, C., Barringer, M., Beattie, S., Kroah-Hartman, G., Frantzen, M., and Lokier, J. Formatguard: automatic protection from printf format string vulnerabilities. In SSYM'01: Proceedings of the 10th conference on USENIX Security Symposium (2001).

Digital Library

[13]

Cowan, C., Beattie, S., Kroah-Hartman, G., Pu, C., Wagle, P., and Gligor, V. Subdomain: Parsimonious server security. In LISA '00: Proceedings of the 14th USENIX conference on System administration (2000).

Digital Library

[14]

Cowan, C., Pu, C., Maier, D., Hintony, H., Walpole, J., Bakke, P., Beattie, S., Grier, A., Wagle, P., and Zhang, Q. Stackguard: automatic adaptive detection and prevention of buffer-overflow attacks. In SSYM'98: Proceedings of the 7th conference on USENIX Security Symposium (1998).

Digital Library

[15]

Devine, S. W., Bugnion, E., and Rosenblum, M. Virtualization system including a virtual machine monitor for a computer with a segmented architecture. US Patent 6397242.

[16]

Fetzer, C., and Suesskraut, M. Switchblade: enforcing dynamic personalized system call models. In Eurosys '08: Proceedings of the 3rd ACM SIGOPS/EuroSys European Conference on Computer Systems 2008 (New York, NY, USA, 2008), ACM, pp. 273--286.

Digital Library

[17]

Ford, B., and Cox, R. Vx32: lightweight user-level sandboxing on the x86. In ATC'08: USENIX 2008 Annual Technical Conference on Annual Technical Conference (Berkeley, CA, USA, 2008), USENIX Association, pp. 293--306.

Digital Library

[18]

Garfinkel, T. Traps and pitfalls: Practical problems in system call interposition based security tools. In In Proc. Network and Distributed Systems Security Symposium (2003), pp. 163--176.

[19]

Garfinkel, T., Pfaff, B., and Rosenblum, M. Ostia: A delegating architecture for secure system call interposition. In Proc. Network and Distributed Systems Security Symposium (February 2004).

[20]

Garfinkel, T., and Rosenblum, M. A virtual machine introspection based architecture for intrusion detection. In Proc. Network and Distributed Systems Security Symposium (February 2003).

[21]

Garg, M. Sysenter based system call mechanism in linux 2.6 (http://manugarg.googlepages.com/systemcallinlinux2\_6.html).

[22]

Goldberg, I., Wagner, D., Thomas, R., and Brewer, E. A. A secure environment for untrusted helper applications: Confining the wily hacker. In Proceedings of the 6th Usenix Security Symposium (1996).

Digital Library

[23]

Hazelwood, K., and Smith, M. D. Managing bounded code caches in dynamic binary optimization systems. TACO '06 3, 3 (2006), 263--294.

Digital Library

[24]

Hiroaki, E., and Kunikazu, Y. propolice : Improved stack-smashing attack detection. IPSJ SIG Notes 2001, 75 (2001-07--25), 181--188.

[25]

Ho, A., Fetterman, M., Clark, C., Warfield, A., and Hand, S. Practical taint-based protection using demand emulation. vol. 40, pp. 29--41.

Digital Library

[26]

Kiriansky, V., Bruening, D., and Amarasinghe, S. P. Secure execution via program shepherding. In Proceedings of the 11th USENIX Security Symposium (Berkeley, CA, USA, 2002), USENIX Association, pp. 191--206.

Digital Library

[27]

Liang, Z., Sun, W., Venkatakrishnan, V. N., and Sekar, R. Alcatraz: An isolated environment for experimenting with untrusted software. ACM Trans. Inf. Syst. Secur. 12, 3 (2009), 1--37.

Digital Library

[28]

Luk, C.-K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Reddi, V. J., and Hazelwood, K. Pin: building customized program analysis tools with dynamic instrumentation. In PLDI '05 (New York, NY, USA, 2005), pp. 190--200.

Digital Library

[29]

McCamant, S., and Morrisett, G. Evaluating SFI for a CISC architecture. In 15th USENIX Security Symposium (Vancouver, BC, Canada, August 2-4, 2006), pp. 209--224.

Digital Library

[30]

PaX-Team. PaX ASLR (Address Space Layout Randomization). http://pax.grsecurity.net/docs/aslr.txt.

[31]

Payer, M., and Gross, T. Requirements for fast binary translation. In 2nd Workshop on Architectural and Microarchitectural Support for Binary Translation (2009).

[32]

Payer, M., and Gross, T. R. Generating low-overhead dynamic binary translators. In SYSTOR'10 (2010).

Digital Library

[33]

Provos, N. Improving host security with system call policies. In SSYM'03: Proceedings of the 12th conference on USENIX Security Symposium (Berkeley, CA, USA, 2003), USENIX Association, pp. 18--18.

Digital Library

[34]

Scott, K., and Davidson, J. Strata: A software dynamic translation infrastructure. Tech. rep., Charlottesville, VA, USA, 2001.

Digital Library

[35]

Scott, K., and Davidson, J. Safe virtual execution using software dynamic translation. Computer Security Applications Conference, Annual 0 (2002), 209.

Digital Library

[36]

Shacham, H., Page, M., Pfaff, B., Goh, E.-J., Modadugu, N., and Boneh, D. On the effectiveness of address-space randomization. In CCS'04 (2004), pp. 298--307.

Digital Library

[37]

Sridhar, S., Shapiro, J. S., and Bungale, P. P. HDTrans: a low-overhead dynamic translator. SIGARCH Comput. Archit. News 35, 1 (2007), 135--140.

Digital Library

[38]

Sridhar, S., Shapiro, J. S., Northup, E., and Bungale, P. P. HDTrans: an open source, low-level dynamic instrumentation system. In VEE '06 (New York, NY, USA, 2006), pp. 175--185.

Digital Library

[39]

Wahbe, R., Lucco, S., Anderson, T. E., and Graham, S. L. Efficient software-based fault isolation. In SOSP'93 (New York, NY, USA, 1993), ACM, pp. 203--216.

Digital Library

[40]

Watson, R. N. M. Exploiting concurrency vulnerabilities in system call wrappers. In WOOT '07: Proceedings of the first USENIX workshop on Offensive Technologies (2007).

Digital Library

[41]

Wright, C., Cowan, C., Smalley, S., Morris, J., and Kroah-Hartman, G. Linux security modules: General security support for the linux kernel. In Proceedings of the 11th USENIX Security Symposium (2002).

Digital Library

[42]

Yee, B., Sehr, D., Dardyk, G., Chen, J. B., Muth, R., Ormandy, T., Okasaka, S., Narula, N., and Fullagar, N. Native client: A sandbox for portable, untrusted x86 native code. IEEE Symposium on Security and Privacy (2009), 79--93.

Digital Library

Cited By

Di Bartolomeo LMoghaddas HPayer MCalandrino JTroncoso C(2023)ARMoreProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620590(6311-6328)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620590
Wruck FSarafov VJakobsmeier FWeiß MGupta MKhorsandroo SAbdelsalam M(2022)GyroidOSProceedings of the 2022 ACM Workshop on Secure and Trustworthy Cyber-Physical Systems10.1145/3510547.3517917(87-96)Online publication date: 18-Apr-2022
https://dl.acm.org/doi/10.1145/3510547.3517917
Kolosick MNarayan SJohnson EWatt CLeMay MGarg DJhala RStefan D(2022)Isolation without taxation: near-zero-cost transitions for WebAssembly and SFIProceedings of the ACM on Programming Languages10.1145/34986886:POPL(1-30)Online publication date: 12-Jan-2022
https://dl.acm.org/doi/10.1145/3498688
Show More Cited By

Index Terms

Fine-grained user-space security through virtualization
1. Security and privacy
  1. Systems security
    1. Operating systems security
2. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Runtime environments

Recommendations

Fine-grained user-space security through virtualization
VEE '11

This paper presents an approach to the safe execution of applications based on software-based fault isolation and policy-based system call authorization. A running application is encapsulated in an additional layer of protection using dynamic binary ...
A comparison of software and hardware techniques for x86 virtualization
ASPLOS XII: Proceedings of the 12th international conference on Architectural support for programming languages and operating systems

Until recently, the x86 architecture has not permitted classical trap-and-emulate virtualization. Virtual Machine Monitors for x86, such as VMware ® Workstation and Virtual PC, have instead used binary translation of the guest kernel code. However, both ...
A comparison of software and hardware techniques for x86 virtualization
Proceedings of the 2006 ASPLOS Conference

Until recently, the x86 architecture has not permitted classical trap-and-emulate virtualization. Virtual Machine Monitors for x86, such as VMware ® Workstation and Virtual PC, have instead used binary translation of the guest kernel code. However, both ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VEE '11: Proceedings of the 7th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

March 2011

250 pages

ISBN:9781450306874

DOI:10.1145/1952682

General Chair:
Erez Petrank
The Technion, Israel
,
Program Chair:
Doug Lea
SUNY Oswego, USA

ACM SIGPLAN Notices Volume 46, Issue 7
VEE '11
July 2011
231 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2007477
Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VEE '11

Sponsor:

VEE '11: ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

March 9 - 11, 2011

California, Newport Beach, USA

Acceptance Rates

Overall Acceptance Rate 80 of 235 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

62
Total Citations
View Citations
801
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Di Bartolomeo LMoghaddas HPayer MCalandrino JTroncoso C(2023)ARMoreProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620590(6311-6328)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620590
Wruck FSarafov VJakobsmeier FWeiß MGupta MKhorsandroo SAbdelsalam M(2022)GyroidOSProceedings of the 2022 ACM Workshop on Secure and Trustworthy Cyber-Physical Systems10.1145/3510547.3517917(87-96)Online publication date: 18-Apr-2022
https://dl.acm.org/doi/10.1145/3510547.3517917
Kolosick MNarayan SJohnson EWatt CLeMay MGarg DJhala RStefan D(2022)Isolation without taxation: near-zero-cost transitions for WebAssembly and SFIProceedings of the ACM on Programming Languages10.1145/34986886:POPL(1-30)Online publication date: 12-Jan-2022
https://dl.acm.org/doi/10.1145/3498688
D’Elia DInvidia LPalmaro FQuerzoni L(2022)Evaluating Dynamic Binary Instrumentation Systems for Conspicuous Features and ArtifactsDigital Threats: Research and Practice10.1145/34785203:2(1-13)Online publication date: 8-Feb-2022
https://dl.acm.org/doi/10.1145/3478520
Tian LShi YChen LYang YShi GLee J(2022)Gadgets splicingProceedings of the 20th IEEE/ACM International Symposium on Code Generation and Optimization10.1109/CGO53902.2022.9741259(155-167)Online publication date: 2-Apr-2022
https://dl.acm.org/doi/10.1109/CGO53902.2022.9741259
Dinesh SBurow NXu DPayer M(2020)RetroWrite: Statically Instrumenting COTS Binaries for Fuzzing and Sanitization2020 IEEE Symposium on Security and Privacy (SP)10.1109/SP40000.2020.00009(1497-1511)Online publication date: May-2020
https://doi.org/10.1109/SP40000.2020.00009
Wang WCui LHao ZFei HWang CPeng Y(2020)pRnR: A Parallel Record-Replay Framework for Virtual Machines2020 IEEE 38th International Conference on Computer Design (ICCD)10.1109/ICCD50377.2020.00106(610-618)Online publication date: Oct-2020
https://doi.org/10.1109/ICCD50377.2020.00106
Mishra SPolychronakis M(2020)Saffire: Context-sensitive Function Specialization against Code Reuse Attacks2020 IEEE European Symposium on Security and Privacy (EuroS&P)10.1109/EuroSP48549.2020.00010(17-33)Online publication date: Sep-2020
https://doi.org/10.1109/EuroSP48549.2020.00010
D'Elia DCoppa ENicchi SPalmaro FCavallaro LGalbraith SRussello GSusilo WGollmann DKirda ELiang Z(2019)SoKProceedings of the 2019 ACM Asia Conference on Computer and Communications Security10.1145/3321705.3329819(15-27)Online publication date: 2-Jul-2019
https://dl.acm.org/doi/10.1145/3321705.3329819
Klemperer PJeon HPayne BHoe J(2019)High-Performance Memory Snapshotting for Real-Time, Consistent, Hypervisor-Based MonitorsIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2018.2805904(1-1)Online publication date: 2019
https://doi.org/10.1109/TDSC.2018.2805904
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten