Abstract
Programmers obfuscate their code to defeat manual or automated analysis. Obfuscations are often used to hide malicious behavior. In particular, malicious programs employ obfuscations of stack-based instructions, such as call and return instructions, to prevent an analyzer from determining which system functions it calls. Instead of using these instructions directly, a combination of other instructions, such as PUSH and POP, are used to achieve the same semantics. This paper presents an abstract interpretation based analysis to detect obfuscation of stack instructions. The approach combines Reps and Balakrishnan’s value set analysis (VSA) and Lakhotia and Kumar’s Abstract Stack Graph, to create an analyzer that can track stack manipulations where the stack pointer may be saved and restored in memory or registers. The analysis technique may be used to determine obfuscated calls made by a program, an important first step in detecting malicious behavior.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Linn, C., Debray, S.: Obfuscation of Executable Code to Improve Resistance to Static Disassembly. In: 10th ACM Conference on Computer and Communications Security, CCS (2003)
Collberg, C., Thomborson, C., Low, D.: A Taxonomy of Obfuscating Transformations. Technical Report 148, Department of Computer Science, University of Auckland (1997)
Lakhotia, A., Kumar, E.U.: Abstract Stack Graph to Detect Obfuscated Calls in Binaries. In: Fourth IEEE International Workshop on Source Code Analysis and Manipulation(SCAM 2004), Chicago, Illinois (2004)
Ször, P., Ferrie, P.: Hunting for Metamorphic. In: Virus Bulletin Conference, Prague, Czech Republic (2001)
Lakhotia, A., Singh, P.K.: Challenges in Getting ’Formal’ with Viruses. Virus Bulletin (2003)
Ször, P.: The New 32-Bit Medusa. Virus Bulletin, 8–10 (2000)
Bergeron, J., Debbabi, M.: Detection of Malicious Code in Cots Software: A Short Survey. In: First International Software Assurance Certification Conference (ISACC 1999), Washington DC (1999)
Symantec, Understanding Heuristics: Symantec’s Bloodhound Technology, http://www.symantec.com/avcenter/reference/heuristc.pdf (Last accessed July 1, 2004)
Balakrishnan, G., Reps, T.: Analyzing Memory Accesses in X86 Executables. In: 13th International Conference on Compiler Construction (2004)
Cousot, P., Cousot, R.: Static Determination of Dynamic Properties of Programs. In: 2nd Int. Symp. on Programming, Dumod, Paris, France (1976)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Venable, M., Chouchane, M.R., Karim, M.E., Lakhotia, A. (2005). Analyzing Memory Accesses in Obfuscated x86 Executables. In: Julisch, K., Kruegel, C. (eds) Detection of Intrusions and Malware, and Vulnerability Assessment. DIMVA 2005. Lecture Notes in Computer Science, vol 3548. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11506881_1
Download citation
DOI: https://doi.org/10.1007/11506881_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26613-6
Online ISBN: 978-3-540-31645-9
eBook Packages: Computer ScienceComputer Science (R0)