Abstract
This paper introduces a new software polymorphism technique that randomizes program data structure layout. This technique will generate different data structure layouts for a program and thus diversify the binary code compiled from the same program source code. This technique can mitigate attacks (e.g., kernel rootkit attacks) that require knowledge about data structure definitions. It is also able to disrupt the generation of data structure-based program signatures. We have implemented our data structure layout randomization technique in the open source compiler collection gcc-4.2.4 and applied it to a number of programs. Our evaluation results show that our technique is able to achieve software binary diversity. We also apply the technique to one operating system data structure in order to foil a number of kernel rootkit attacks. Meanwhile, programs produced by the technique were analyzed by a state-of-the-art data structure inference system and it was demonstrated that reliance on data structure signatures alone may lead to false negatives in malware detection.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Gnu compiler collection (gcc) internals, http://gcc.gnu.org/onlinedocs/gccint/
Offensive computing, http://www.offensivecomputing.net/
Using the gnu compiler collection (gcc), http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gcc/
Vx heavens, http://vx.netlux.org/
Aho, A.V., Lam, M.S., Sethi, R., Ullman, J.D.: Compilers: Principles, Techniques and Tools, 2nd edn. Addison-Wesley, Reading (2006)
Barrantes, E.G., Ackley, D.H., Palmer, T.S., Stefanovic, D., Zovi, D.D.: Randomized instruction set emulation to disrupt binary code injection attacks. In: Proceedings of the 10th ACM conference on Computer and communications security (CCS 2003), pp. 281–289. ACM, New York (2003)
Berger, E.D., Zorn, B.G.: Diehard: probabilistic memory safety for unsafe languages. In: Proceedings of the 2006 ACM SIGPLAN conference on Programming language design and implementation (PLDI 2006), pp. 158–168. ACM, New York (2006)
Bhatkar, E., Duvarney, D.C., Sekar, R.: Address obfuscation: an efficient approach to combat a broad range of memory error exploits. In: Proceedings of the 12th USENIX Security Symposium, pp. 105–120 (2003)
Bhatkar, S., Sekar, R.: Data space randomization. In: Zamboni, D. (ed.) DIMVA 2008. LNCS, vol. 5137, pp. 1–22. Springer, Heidelberg (2008)
Bhatkar, S., Sekar, R., DuVarney, D.C.: Efficient techniques for comprehensive protection from memory error exploits. In: Proceedings of the 14th conference on USENIX Security Symposium, Berkeley, CA, USA (2005), USENIX Association
Caballero, J., Yin, H., Liang, Z., Song, D.: Polyglot: Automatic extraction of protocol format using dynamic binary analysis. In: Proceedings of the 14th ACM Conference on Computer and and Communications Security (CCS 2007) (2007)
Cadar, C., Akritidis, P., Costa, M., Martin, J.-P., Castro, M.: Data randomization. Technical Report MSR-TR-2008-120, Microsoft Research (2008)
Chew, M., Song, D.: Mitigating buffer overflows by operating system randomization. Technical Report CMU-CS-02-197, Carnegie Mellon University (2002)
Cho, S., Chang, H., Cho, Y.: Implementation of an obfuscation tool for c/c++ source code protection on the xscale architecture. In: Brinkschulte, U., Givargis, T., Russo, S. (eds.) SEUS 2008. LNCS, vol. 5287, pp. 406–416. Springer, Heidelberg (2008)
Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations (1997)
Comparetti, P.M., Wondracek, G., Kruegel, C., Kirda, E.: Prospex: Protocol specification extraction. In: Proceedings of 2009 IEEE Symposium on Security and Privacy, Oakland, CA (May 2009)
Cowan, C., Beattie, S., Johansen, J., Wagle, P.: Pointguard: protecting pointers from buffer overflow vulnerabilities. In: Proceedings of the 12th conference on USENIX Security Symposium, Berkeley, CA, USA (2003), USENIX Association
Cox, B., Evans, D., Filipi, A., Rowanhill, J., Hu, W., Davidson, J., Knight, J., Nguyen-Tuong, A., Hiser, J.: N-variant systems: a secretless framework for security through diversity. In: Proceedings of the 15th conference on USENIX Security Symposium, Berkeley, CA, USA (2006), USENIX Association
Cozzie, A., Stratton, F., Xue, H., King, S.T.: Digging for data structures. In: Proceeding of 8th Symposium on Operating System Design and Implementation (OSDI 2008) (December 2008)
Cui, W., Peinado, M., Chen, K., Wang, H.J., Irun-Briz, L.: Tupni: Automatic reverse engineering of input formats. In: Proceedings of the 15th ACM Conference on Computer and Communications Security (CCS 2008), Alexandria, VA (October 2008)
Cui, W., Peinado, M., Wang, H.J., Locasto, M.: Shieldgen: Automatic data patch generation for unknown vulnerabilities with informed probing. In: Proceedings of 2007 IEEE Symposium on Security and Privacy, Oakland, CA (May 2007)
Etoh, H.: GCC extension for protecting applications from stack-smashing attacks, ProPolice (2003), http://www.trl.ibm.com/projects/security/ssp/
Forrest, S., Somayaji, A., Ackley, D.: Building diverse computer systems. In: Proceedings of the 6th Workshop on Hot Topics in Operating Systems (HotOS-VI), Washington, DC, USA, p. 67. IEEE Computer Society, Los Alamitos (1997)
Garfinkel, T., Rosenblum, M.: A Virtual Machine Introspection Based Architecture for Intrusion Detection. In: Proc. Network and Distributed Systems Security Symposium (NDSS 2003) (February 2003)
Golovanevsky, O., Zaks, A.: Struct-reorg: current status and future perspectives. In: Proceedings of the GCC Developers’ Summit (2007)
Hagog, M., Tice, C.: Cache aware data layout reorganization optimization in gcc. In: Proceedings of the GCC Developers’ Summit (2005)
Jiang, X., Wang, H.J., Xu, D., Wang, Y.-M.: Randsys: Thwarting code injection attacks with system service interface randomization. In: Proceedings of the 26th IEEE International Symposium on Reliable Distributed Systems (SRDS 2007), Washington, DC, USA, pp. 209–218. IEEE Computer Society, Los Alamitos (2007)
Kc, G.S., Keromytis, A.D., Prevelakis, V.: Countering code-injection attacks with instruction-set randomization. In: Proceedings of the 10th ACM conference on Computer and communications security (CCS 2003), Washington D.C., USA, pp. 272–280. ACM, New York (2003)
Lin, Z., Jiang, X., Xu, D., Zhang, X.: Automatic protocol format reverse engineering through context-aware monitored execution. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008), San Diego, CA (February 2008)
Lin, Z., Zhang, X.: Deriving input syntactic structure from execution. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2008), Atlanta, GA, USA (November 2008)
Low, D.: Protecting java code via code obfuscation. Crossroads 4(3), 21–23 (1998)
Novark, G., Berger, E.D., Zorn, B.G.: Exterminator: automatically correcting memory errors with high probability. In: Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 007), San Diego, California, USA. ACM Press, New York (2007)
Percival, C.: Naive differences of executable code (2003), http://www.daemonology.net/bsdiff/
Raman, E., Hundt, R., Mannarswamy, S.: Structure layout optimization for multithreaded programs. In: Proceedings of the International Symposium on Code Generation and Optimization (CGO 2007), Washington, DC, USA, pp. 271–282. IEEE Computer Society Press, Los Alamitos (2007)
Riley, R., Jiang, X., Xu, D.: Guest-Transparent Prevention of Kernel Rootkits with VMM-Based Memory Shadowing. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 1–20. Springer, Heidelberg (2008)
Salamat, B., Gal, A., Yermolovich, A., Manivannan, K., Franz, M.: Reverse stack execution. Technical Report No. 07-07, University of California, Irvine (2007)
Seshadri, A., Luk, M., Qu, N., Perrig, A.: SecVisor: A Tiny Hypervisor to Guarantee Lifetime Kernel Code Integrity for Commodity OSes. In: Proceedings of the ACM Symposium on Operating Systems Principles (SOSP 2007) (October 2007)
PaX Team. Pax address space layout randomization (aslr), http://pax.grsecurity.net/docs/aslr.txt
Wang, X., Li, Z., Xu, J., Reiter, M.K., Kil, C., Choi, J.Y.: Packet vaccine: Black-box exploit detection and signature generation. In: Proceedings of the 13th ACM Conference on Computer and Communication Security (CCS 2006), pp. 37–46. ACM Press, New York (2006)
Wondracek, G., Milani, P., Kruegel, C., Kirda, E.: Automatic network protocol analysis. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008), San Diego, CA (February 2008)
Xu, J., Kalbarczyk, Z., Iyer, R.K.: Transparent runtime randomization for security. In: Proceedings of the 22nd International Symposium on Reliable Distributed Systems (SRDS 2003), pp. 260–269. IEEE Computer Society, Los Alamitos (2003)
Zhong, Y., Orlovich, M., Shen, X., Ding, C.: Array regrouping and structure splitting using whole-program reference affinity. In: Proceedings of the ACM SIGPLAN 2004 conference on Programming language design and implementation, PLDI 2004 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lin, Z., Riley, R.D., Xu, D. (2009). Polymorphing Software by Randomizing Data Structure Layout. In: Flegel, U., Bruschi, D. (eds) Detection of Intrusions and Malware, and Vulnerability Assessment. DIMVA 2009. Lecture Notes in Computer Science, vol 5587. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02918-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-02918-9_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02917-2
Online ISBN: 978-3-642-02918-9
eBook Packages: Computer ScienceComputer Science (R0)