Abstract
Self modifying code (SMC) are code snippets that modify themselves at runtime. Malware use SMC to hide payloads and achieve persistence. Software-based SMC detection solutions impose performance penalties for real-time monitoring and do not benefit from runtime architectural information (cache invalidation or pipeline flush, for instance). We revisit SMC impact on hardware internals and discuss the implementation of an SMC detector at distinct architectural points. We consider three detection approaches: (i) existing hardware counters; (ii) block invalidation by the cache coherence protocol; (iii) the use of Memory Management Unit (MMU) information to control SMC execution. We compare the identified instrumentation points to highlight their strong and weak points. We also compare them to previous SMC detectors’ implementations.
Similar content being viewed by others
Notes
We are hereafter referring to overhead to denote the runtime monitoring overhead, as the overhead of running detection routines is unavoidable to any AV solution.
References
AMD: AMD Secure Virtual Machine Architecture Reference Manual (2008). https://www.mimuw.edu.pl/~vincent/lecture6/sources/amd-pacifica-specification.pdf
ARM: Smc (2013). https://community.arm.com/processors/b/blog/posts/caches-and-self-modifying-code
Babar, K., Khalid, F.: Generic unpacking techniques. In: International Conference on Computer, Control and Communication, pp. 1–6 (2009)
Ballapuram, C.S., Sharif, A., Lee, H.H.S.: Exploiting access semantics and program behavior to reduce snoop power in chip multiprocessors. SIGARCH Comput. Archit. News 36(1), 60–69 (2008)
Bonfante, G., Fernandez, J., Marion, J.Y., Rouxel, B., Sabatier, F., Thierry, A.: Codisasm: Medium scale concatic disassembly of self-modifying binaries with overlapping instructions. In: ACM SIGSAC Conference on Computer and Communications Security, pp. 745–756 (2015)
Borello, J.M., Mé, L.: Code obfuscation techniques for metamorphic viruses. JICVHT 3, 211–220 (2008)
Botacin, M., de Geus, P., Grégio, A.: Enhancing branch monitoring for security purposes: from control flow integrity to malware analysis and debugging. ACM Trans. Priv. Secur. 21(1), 1–30 (2018)
Cai, H., Shao, Z., Vaynberg, A.: Certified self-modifying code. SIGPLAN Not. 42(6), 66–77 (2007)
Caserta, P., Zendra, O.: A tracing technique using dynamic bytecode instrumentation of java applications and libraries at basic block level. In: Proceedings of the 6th Workshop on Implementation, Compilation, Optimization of Object-Oriented Languages, Programs and Systems, ICOOOLPS’11, pp. 6:1–6:4. ACM, New York, NY, USA (2011). https://doi.org/10.1145/2069172.2069178
Censier, F.: A new solution to coherence problems in multicache systems. IEEE Trans. Comput. C–27(12), 1112–1118 (1978). https://doi.org/10.1109/TC.1978.1675013
Coke, J., Baliga, H., Cooray, N., Gamsaragan, E., Smith, P., Yoon, K., Abel, J., Valles., A.: Improvements in the Intel’s Core2 penryn processor family architecture and microarchitecture (2008)
Debray, S., Patel, J.: Reverse engineering self-modifying code: unpacker extraction. In: Working Conference on Reverse Engineering, pp. 131–140 (2010)
Dehnert, J., Grant, B., Banning, J., Johnson, R., Kistler, T., Klaiber, A., Mattson, J.: The transmeta code morphing trade; software: using speculation, recovery, and adaptive retranslation to address real-life challenges. In: International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization, pp. 15–24. IEEE Computer Society (2003)
Dinaburg, A., Royal, P., Sharif, M., Lee, W.: Ether: malware analysis via hardware virtualization extensions. In: ACM Conference on Computer and Communications Security, pp. 51–62 (2008)
Gebai, M., Dagenais, M.R.: Survey and analysis of kernel and userspace tracers on linux: design, implementation, and overhead. ACM Comput. Surv. 51(2), 26:1–26:33 (2018). https://doi.org/10.1145/3158644
Gruss, D., Maurice, C., Wagner, K., Mangard, S.: Flush+flush: a fast and stealthy cache attack. In: Interenational Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, pp. 279–299 (2016)
Gutierrez, A., Pusdesris, J., Dreslinski, R.G., Mudge, T.: Lazy cache invalidation for self-modifying codes. In: International Conference on Compilers, Architectures and Synthesis for Embedded Systems, pp. 151–160 (2012)
Intel: Intel 64 and IA-32 Architectures Software Developer’s Manual. Intel (2013). https://www.intel.com.br/content/www/br/pt/architecture-and-technology/64-ia-32-architectures-software-developersystem-programming-manual-325384.html
Inci, M.S., Eisenbarth, T., Sunar, B.: Wait a Minute! A Fast, Cross-VM Attack on AES, pp. 299–319. Springer, Springer (2014)
Korczynski, D.: Repeconstruct: reconstructing binaries with self-modifying code and import address table destruction. In: International Conference on Malicious and Unwanted Software, pp. 1–8 (2016)
Liu, A., Wang, W.: Ascms: an accurate self-modifying code cache management strategy in binary translation. In: International Conference on Information Science and Cloud Computing Companion, pp. 405–410 (2013)
Luk, C.K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Reddi, V.J., Hazelwood, K.: Pin: building customized program analysis tools with dynamic instrumentation. In: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI’05, pp. 190–200. ACM, New York, NY, USA (2005). https://doi.org/10.1145/1065010.1065034
Maebe, J., De Bosschere, K.: Self-modifying Code (2003). arXiv:cs/0309029
Martignoni, L., Christodorescu, M., Jha, S.: Omniunpack: fast, generic, and safe unpacking of malware. In: Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007), pp. 431–441 (2007). https://doi.org/10.1109/ACSAC.2007.15
Microsoft: x64 intrinsics. https://msdn.microsoft.com/en-us/library/hh977022.aspx
Mody, R.P.: Functional programming is not self-modifying code. SIGPLAN Not. 27(11), 13–14 (1992)
Moret, P., Binder, W., Tanter, E.: Polymorphic bytecode instrumentation. In: Proceedings of the Tenth International Conference on Aspect-oriented Software Development, AOSD’11, pp. 129–140. ACM, New York, NY, USA (2011). https://doi.org/10.1145/1960275.1960292
Neiger, G.,, Santoni, A., Leung, F., Rodgers, D., Uhlig, R.: Intel virtualization technology: hardware support for efficient processor virtualization (2006). https://www.ece.cmu.edu/~ece845/docs/vt-overview-itj06.pdf. Accessed Jan 2019
Ray, K., Kramer, M., England, P., Field, S.: On-access scan of memory for malware, US Patent 7,836,504 (2010)
Roemer, R., Buchanan, E., Shacham, H., Savage, S.: Return-oriented programming: systems, languages, and applications. ACM Trans. Inf. Syst. Secur. 15(1), 1–34 (2012)
Shar, L.E., Lawton, K.P.: Trace cache for efficient self-modifying code processing, US Patent 7,606,975 (2009)
Uluski, D., Moffie, M., Kaeli, D.: Characterizing antivirus workload execution. SIGARCH Comput. Archit. News 33(1), 90–98 (2005). https://doi.org/10.1145/1055626.1055639
Willems, C., Hund, R., Fobian, A., Felsch, D., Holz, T., Vasudevan, A.: Down to the bare metal: using processor features for binary analysis. In: ACSAC, pp. 189–198 (2012)
Wu, M., Zhang, Y., Mi, X.: Binary protection using dynamic fine-grained code hiding and obfuscation. In: International Conference on Information and Network Security, pp. 1–8 (2016)
Xianya, M., Yi, Z., Baosheng, W., Yong, T.: A survey of software protection methods based on self-modifying code. In: International Conference on Computational Intelligence and Communication Networks, pp. 589–593 (2015)
Yarom, Y., Falkner, K.: Flush+reload: a high resolution, low noise, l3 cache side-channel attack. In: USENIX Security, pp. 719–732 (2014)
Zaidi, N.: System and method for tracking in-flight instructions in a pipeline, US Patent 6,237,088 (2001)
Acknowledgements
This project was partially financed by the Serrapilheira Institute (Grant Number Serra-1709-16621), the Brazilian National Counsel of Technological and Scientific Development (CNPq, Ph.D. Scholarship, process 164745/2017-3) and the Coordination for the Improvement of Higher Education Personnel (CAPES, Project FORTE, Forensics Sciences Program 24/2014, process 23038.007604/2014-69).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Botacin, M., Zanata, M. & Grégio, A. The self modifying code (SMC)-aware processor (SAP): a security look on architectural impact and support. J Comput Virol Hack Tech 16, 185–196 (2020). https://doi.org/10.1007/s11416-020-00348-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11416-020-00348-w