Skip to main content

Static Analysis on x86 Executables for Preventing Automatic Mimicry Attacks

  • Conference paper
Detection of Intrusions and Malware, and Vulnerability Assessment (DIMVA 2007)

Abstract

In 2005, Kruegel et al. proposed a variation of the traditional mimicry attack, to which we will refer to as automatic mimicry, which can defeat existing system call based HIDS models. We show how such an attack can be defeated by using information provided by the Interprocedural Control Flow Graph (ICFG). Roughly speaking, by exploiting the ICFG of a protected binary, we propose a strategy based on the use of static analysis techniques which is able to localize critical regions inside a program, which are segments of code that could be used for exploiting an automatic mimicry attack. Once the critical regions have been recognized, their code is instrumented in such a way that, during the executions of such regions, the integrity of the dangerous code pointers is monitored, and any unauthorized modification will be restored at once with the legal values. Moreover, our experiments shows that such a defensive mechanism presents a low run-time overhead.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abadi, M., Budiu, M., Erlingsson, U., Ligatti, J.: Control-flow integrity. In: CCS 2005: Proceedings of the 12th ACM conference on Computer and communications security, pp. 340–353. ACM Press, New York (2005)

    Chapter  Google Scholar 

  2. appel, a.w.: Modern compiler implementation in c. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  3. Balakrishnan, G., Reps, T.: Analyzing memory accesses in x86 executables. In: Duesterwald, E. (ed.) CC 2004. LNCS, vol. 2985, pp. 5–23. Springer, Heidelberg (2004)

    Google Scholar 

  4. Cox, B., Evans, D., Filipi, A., Rowanhill, J., Hu, W., Davidson, J., Knight, J., Nguyen-Tuong, A., Hiser, J.: N-Variant Systems: A Secretless Framework for Security through Diversity. In: 15th USENIX Security Symposium (2006)

    Google Scholar 

  5. Bruschi, D., Cavallaro, L., Lanzi, A.: An Efficient Technique for Preventing Mimicry and Impossible Paths Execution Attacks. In: 3rd International Workshop on Information Assurance (WIA 2007) (April 2007)

    Google Scholar 

  6. Chen, S., Xu, J., Sezer, E., Gauriar, P., Iye, R.K.: Non-Control-Data Attacks Are Realistic Threats. In: 14th USENIX Security Symposium (2005)

    Google Scholar 

  7. Cowan, C., Pu, C., Maier, D., Walpole, J., Bakke, P., Beattie, S., Grier, A., Wagle, P., Zhang, Q., Hinton, H.: StackGuard: Automatic adaptive detection and prevention of buffer-overflow attacks. In: Proc. of the 7th Usenix Security Symposium, pp. 63–78 (January 1998)

    Google Scholar 

  8. Cytron, R., Ferrante, J., Rosen, B.K., Wegman, M.N., Zadeck, F.K.: Efficiently computing static single assignment form and the control dependence graph. ACM Trans. Program. Lang. Syst. 13(4), 451–490 (1991)

    Article  Google Scholar 

  9. Bruschi, D., Cavallaro, L., Lanzi, A.: Diversified Process Replicæ for Defeating Memory Error Exploits. In: 3rd International Workshop on Information Assurance (WIA 2007) (April 2007)

    Google Scholar 

  10. Etoh, H.: GCC extension for protecting applications from stack-smashing attacks (ProPolice) (2003), http://www.trl.ibm.com/projects/security/ssp/

  11. Feng, H., Kolesnikov, O., Fogla, P., Lee, W., Gong, W.: Anomaly Detection using Call Stack Information. In: IEEE Symposium on Security and Privacy, Oakland, California (2003)

    Google Scholar 

  12. Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A Sense of Self for Unix Processes. In: SP 1996: Proceedings of the 1996 IEEE Symposium on Security and Privacy, p. 120. IEEE Computer Society Press, Los Alamitos (1996)

    Chapter  Google Scholar 

  13. Hofmeyr, S.A., Forrest, S., Somayaji, A.: Intrusion Detection Using Sequences of System Calls. Journal of Computer Security 6(3), 151–180 (1998)

    Google Scholar 

  14. Shacham, H., Page, M., Pfaff, B., Goh, E.-J.: On the Effectiveness of Address-Space Randomization. In: CCS 2004: Proceedings of the 11th ACM Conference on Computer and Communications Security, pp. 298–307. ACM Press, New York (2004)

    Chapter  Google Scholar 

  15. iSec.pl Development Team. kNoX - Implementation of non-executable Page Protection Mechanism (February 2005), http://www.isec.pl/projects/knox/knox.html

  16. Kiriansky, V., Bruening, D., Amarasinghe, S.P.: Secure execution via program shepherding. In: Proceedings of the 11th USENIX Security Symposium, pp. 191–206, Berkeley, CA, USA, USENIX Association (2002)

    Google Scholar 

  17. Kruegel, C., Kirda, E., Mutz, D., Robertson, W., Vigna, G.: Automating Mimicry Attacks Using Static Binary Analysis. In: Proceedings of the USENIX Security Symposium, Baltimore, MD (August 2005)

    Google Scholar 

  18. Elias Aleph One Levy. Smashing the Stack for Fun and Profit. Phrack Magazine, vol. 0x07(#49), Phile 14–16 (December 1998)

    Google Scholar 

  19. Nielson, F., Nielson, H., Hankin, C.: Principles of Program Analysis (1999)

    Google Scholar 

  20. Bhatkar, S., DuVarney, D.C., Sekar, R.: Address Obfuscation: An Efficient Approach to Combat a Broad Range of Memory Error Exploits. In: 12th USENIX Security Symposium (2003)

    Google Scholar 

  21. Bhatkar, S., Sekar, R., DuVarney, D.C.: Efficient Techniques for Comprehensive Protection from Memory Error Exploits. In: 14th USENIX Security Symposium (2005)

    Google Scholar 

  22. Schwarz, B., Debray, S., Andrews, G.: Disassembly of Executable Code Revisited. In: Proceedings of the Ninth Working Conference on Reverse Engineering (2002)

    Google Scholar 

  23. Sekar, R., Bendre, M., Dhurjati, D., Bollineni, P.: A Fast Automaton-Based Method for Detecting Anomalous Program Behaviors. In: IEEE Symposium on Security and Privacy, Oakland, California (2001)

    Google Scholar 

  24. De Sutter, B., De Bus, B., De Bosschere, K., Keyngnaert, P., Demoen, B.: the static analysis of indirect control transfers in binaries. In: Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications, Las Vegas, Nevada, USA, pp. 1013–1019 (June 2000)

    Google Scholar 

  25. Tan, K.M.C., Killourhy, K.S., Maxion, R.A.: Undermining an anomaly-based intrusion detection system using common exploits. In: Proceedings of the 5th International Symposium on Recent Advances in Intrusion Detection (2002)

    Google Scholar 

  26. Tan, K.M.C., McHugh, J., Killourhy, K.S.: Hiding intrusions: From the abnormal to the normal and beyond. In: Information Hiding, pp. 1–17 (2002)

    Google Scholar 

  27. The Linux Kernel 2.6 Development Team. The Linux Kernel 2.6 (February 2005), http://lwn.net/Articles/121845/

  28. The OpenWall Development Team. The OpenWall Project (February 2005), http://www.openwall.com

  29. The PaX Team. PaX: Address Space Layout Randomization (ASLR), http://pax.grsecurity.net

  30. Wagner, D., Dean, D.: Intrusion Detection via Static Analysis. In: IEEE Symposium on Security and Privacy, Oakland, California (2001)

    Google Scholar 

  31. Wagner, D., Soto, P.: Mimicry Attacks on Host Based Intrusion Detection Systems. In: Proc. Ninth ACM Conference on Computer and Communications Security (2002)

    Google Scholar 

  32. Xu, H., Du, W., Chapin, S.J.: Context Sensitive Anomaly Monitoring of Process Control Flow to Detect Mimicry Attacks and Impossible Paths. In: Jonsson, E., Valdes, A., Almgren, M. (eds.) RAID 2004. LNCS, vol. 3224, pp. 21–38. Springer, Heidelberg (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Bernhard M. Hämmerli Robin Sommer

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bruschi, D., Cavallaro, L., Lanzi, A. (2007). Static Analysis on x86 Executables for Preventing Automatic Mimicry Attacks. In: M. Hämmerli, B., Sommer, R. (eds) Detection of Intrusions and Malware, and Vulnerability Assessment. DIMVA 2007. Lecture Notes in Computer Science, vol 4579. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73614-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-73614-1_13

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-73613-4

  • Online ISBN: 978-3-540-73614-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics