skip to main content
10.1145/2151024.2151053acmconferencesArticle/Chapter ViewAbstractPublication PagesveeConference Proceedingsconference-collections
research-article

V2E: combining hardware virtualization and softwareemulation for transparent and extensible malware analysis

Published:03 March 2012Publication History

ABSTRACT

A transparent and extensible malware analysis platform is essential for defeating malware. This platform should be transparent so malware cannot easily detect and bypass it. It should also be extensible to provide strong support for heavyweight instrumentation and analysis efficiency. However, no existing platform can meet both requirements. Leveraging hardware virtualization technology, analysis platforms like Ether can achieve good transparency, but its instrumentation support and analysis efficiency is poor. In contrast, software emulation provides strong support for code instrumentation and good analysis efficiency by using dynamic binary translation. However, analysis platforms based on software emulation can be easily detected by malware and thus is poor in transparency. To achieve both transparency and extensibility, we propose a new analysis platform that combines hardware virtualization and software emulation. The essence is precise heterogeneous replay: the malware execution is recorded via hardware virtualization and then replayed in software. Our design ensures the execution replay is precise. Moreover, with page-level recording granularity, the platform can easily adjust to analyze various forms of malware (a process, a kernel module, or a shared library). We implemented a prototype called V2E and demonstrated its capability and efficiency by conducting an extensive evaluation with both synthetic samples and 14 realworld emulation-resistant malware samples.

References

  1. adore-ng. adore-ng rootkit. http://stealth.openwall.net/rootkits/.Google ScholarGoogle Scholar
  2. anubis. Anubis: Analyzing Unknown Binaries. http://anubis.iseclab.org/.Google ScholarGoogle Scholar
  3. D. Balzarotti, M. Cova, C. Karlberger, C. Kruegel, E. Kirda, and G. Vigna. Efficient Detection of Split Personalities in Malware. In Proceedings of the Network and Distributed System Security Symposium (NDSS), San Diego, CA, February 2010.Google ScholarGoogle Scholar
  4. U. Bayer, P. Comparetti, C. Hlauschek, C. Kruegel, and E. Kirda. Scalable, behavior-based malware clustering. In Proceedings of the 16th Annual Network and Distributed System Security Symposium (NDSS 2009), 2009.Google ScholarGoogle Scholar
  5. F. Bellard. Qemu, a fast and portable dynamic translator. In USENIX Annual Technical Conference, FREENIX Track, April 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. D. Bruening, T. Garnett, and S. Amarasinghe. An infrastructure for adaptive dynamic optimization. In International Symposium on Code Generation and Optimization (CGO'03), March 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. J. Caballero, H. Yin, Z. Liang, and D. Song. Polyglot: Automatic extraction of protocol message format using dynamic binary analysis. In Proceedings of the 14th ACM Conference on Computer and Communication Security (CCS'07), October 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. J. Caballero, P. Poosankam, C. Kreibich, and D. Song. Dispatcher: Enabling active botnet infiltration using automatic protocol reverse-engineering. In Proceedings of the 16th ACM Conference on Computer and Communication Security (CCS'09), Chicago, IL, Nov. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. J. Caballero, N. M. Johnson, S. McCamant, and D. Song. Binary code extraction and interface identification for security applications. In Proceedings of the 17th Annual Network and Distributed System Security Symposium (NDSS'10), San Diego, CA, Feb. 2010.Google ScholarGoogle Scholar
  10. J. Chow, T. Garfinkel, and P. Chen. Decoupling dynamic program analysis from execution in virtual environments. In Proceedings of 2008 Usenix Annual Technical Conference (ATC'08), June 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. cwsandbox. CWSandbox::Behavior-based Malware Analysis. http://mwanalysis.org/.Google ScholarGoogle Scholar
  12. C. da Wang and S. Ju. The dilemma of covert channels searching. In Information Security and Cryptology (ICISC'05), pages 169--174, 2005. Google ScholarGoogle Scholar
  13. A. Dinaburg, P. Royal, M. Sharif, and W. Lee. Ether: malware analysis via hardware virtualization extensions. In Proceedings of the 15th ACM Conference on Computer and Communications Security, pages 51--62, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. G. W. Dunlap, S. T. King, S. Cinar, M. A. Basrai, and P. M. Chen. Revirt: Enabling intrusion analysis through virtual-machine logging and replay. In Proceedings of the 5th symposium on Operating Systems Design and Implementation (OSDI'02), December 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. M. Egele, C. Kruegel, E. Kirda, H. Yin, and D. Song. Dynamic Spyware Analysis. In Proceedings of the 2007 Usenix Annual Technical Conference (ATC'07), June 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. P. Ferrie. Attacks on virtual machine emulators. Symantec Security Response, December 2006.Google ScholarGoogle Scholar
  17. T. Garfinkel and M. Rosenblum. A virtual machine introspection based architecture for intrusion detection. In Proceedings of Network and Distributed Systems Security Symposium (NDSS'03), February 2003.Google ScholarGoogle Scholar
  18. D. Geels, G. Altekar, S. Shenker, and I. Stoica. Replay debugging for distributed applications. In Proceedings of the 2006 USENIX Annual Technical Conference (ATC'06), pages 27--27, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Z. Guo, X. Wang, J. Tang, X. Liu, Z. Xu, M. Wu, M. F. Kaashoek, and Z. Zhang. R2: An application-level kernel for record and replay. In Proceedings of the 9th Symposium on Operating Systems Design and Implementation (OSDI'08), pages 193--208, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. X. Jiang, X. Wang, and D. Xu. Stealthy malware detection through vmm-based "out-of-the-box" semantic view reconstruction. In Proceedings of the 14th ACM conference on Computer and Communications Security (CCS'07), October 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. M. G. Kang, P. Poosankam, and H. Yin. Renovo: A hidden code extractor for packed executables. In Proceedings of the 5th ACM Workshop on Recurring Malcode (WORM'07), Oct. 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. M. G. Kang, H. Yin, S. Hanna, S. McCamant, and D. Song. Emulating emulation-resistant malware. In Proceedings of the 2nd Workshop on Virtual Machine Security (VMSec'09), November 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. kvm. Kernel Based Virtual Machine. http://www.linux-kvm.org/.Google ScholarGoogle Scholar
  24. A. Lanzi, M. Sharif, and W. Lee. K-Tracer: A system for extracting kernel malware behavior. In Proceedings of the 16th Annual Network and Distributed System Security Symposium (NDSS'09), February 2009.Google ScholarGoogle Scholar
  25. C.-K. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S. Wallace, V. J. Reddi, and K. Hazelwood. Pin: Building customized program analysis tools with dynamic instrumentation. In Proceedings of the 2005 ACM SIGPLAN conference on Programming Language Design and Implementation (PLDI'05), june 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. L. Martignoni, R. Paleari, G. F. Roglia, and D. Bruschi. Testing cpu emulators. In Proceedings of the 18th International Symposium on Software Testing and Analysis (ISSTA'09), pages 261--272, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. A. Moser, C. Kruegel, and E. Kirda. Exploring multiple execution paths for malware analysis. In Proceedings of the 2007 IEEE Symposium on Security and Privacy(Oakland'07), May 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. N. Nethercote and J. Seward. Valgrind: a framework for heavyweight dynamic binary instrumentation. In Proceedings of the 2007 ACM SIGPLAN conference on Programming Language Design and Implementation (PLDI'07), pages 89--100, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. M. Olszewski, J. Ansel, and S. Amarasinghe. Kendo: efficient deterministic multithreading in software. Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'09), Mar. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. qemulink. Qemu. http://fabrice.bellard.free.fr/qemu/.Google ScholarGoogle Scholar
  31. T. Raffetseder, C. Krügel, and E. Kirda. Detecting system emulators. In the 10th Information Security Conference (ISC'07), pages 1--18, October 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. R. Riley, X. Jiang, and D. Xu. Multi-aspect profiling of kernel rootkit behavior. In Proceedings of the fourth ACM european conference on Computer systems (EuroSys'09), 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Y. Saito. Jockey: a user-space library for record-replay debugging. In Proceedings of the sixth International Symposium on Automated Analysis-driven Debugging (AADEBUG'05), pages 69--76, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. M. Sharif, A. Lanzi, J. Giffin, and W. Lee. Automatic reverse engineering of malware emulators. In Proceedings of the 30th IEEE Symposium on Security and Privacy (Oakland'09), pages 94--109, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. M. Siper. Introduction to the Theory of Computation. International Thomson Publishing, 1996.Google ScholarGoogle Scholar
  36. S. M. Srinivasan, S. Kandula, C. R. Andrews, and Y. Zhou. Flashback: a lightweight extension for rollback and deterministic replay for software debugging. In Proceedings of the 2004 USENIX Annual Technical Conference (ATC'04), June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. temu. TEMU: The BitBlaze dynamic analysis component. http://bitblaze.cs.berkeley.edu/temu.html.Google ScholarGoogle Scholar
  38. A. Vasudevan and R. Yerraballi. Cobra: Fine-grained malware analysis using stealth localized-executions. In Proceedings of the 2006 IEEE Symposium on Security and Privacy (Oakland'06), pages 264--279, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. vmware. Vmware. http://www.vmware.com/.Google ScholarGoogle Scholar
  40. H. Yin, D. Song, E. Manuel, C. Kruegel, and E. Kirda. Panorama: Capturing system-wide information flow for malware detection and analysis. In Proceedings of the 14th ACM Conference on Computer and Communication Security (CCS'07), October 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. H. Yin, Z. Liang, and D. Song. HookFinder: Identifying and understanding malware hooking behaviors. In Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS'08), February 2008.Google ScholarGoogle Scholar

Index Terms

  1. V2E: combining hardware virtualization and softwareemulation for transparent and extensible malware analysis

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        VEE '12: Proceedings of the 8th ACM SIGPLAN/SIGOPS conference on Virtual Execution Environments
        March 2012
        248 pages
        ISBN:9781450311762
        DOI:10.1145/2151024
        • cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 47, Issue 7
          VEE '12
          July 2012
          229 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/2365864
          Issue’s Table of Contents

        Copyright © 2012 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 3 March 2012

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        Overall Acceptance Rate80of235submissions,34%

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader