Abstract
Sandbox technique is widely used in automated malware analysis. However, it can only see one path during its analysis. This is fatal when meeting the targeted malware. The challenge is how to unleash the hidden behaviors of targeted malware. Many works have been done to mitigate this problem. However, these solutions either use limited and fixed sandbox environments or introduce time and space consuming multi-path exploration. To address this problem, this paper proposes a new hybrid dynamic analysis scheme by applying function summary based symbolic execution of malware. Specifically, by providing Windows APIs’ summary stub and using unicorn CPU emulator, we can effectively extract malware’s hidden behavior which are not shown in sandbox environment. Without the usage of full system emulation, our approach achieve much higher speed than existing schemes. We have implemented a prototype system, and evaluated it with typical real-world malware samples. The experiment results show that our system can effectively and efficiently extract malware’s hidden behavior.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balzarotti, D., Cova, M., Karlberger, C., Kruegel, C., Kirda, E., Vigna, G.: Efficient detection of split personalities in malware. In: NDSS 2010, 17th Annual Network and Distributed System Security Symposium, February 2010
Bayer, U., Moser, A., Kruegel, C., Kirda, E.: Dynamic analysis of malicious code. J. Comput. Virol. 2(1), 67–77 (2006)
Bilge, L., Dumitras, T.: Before we knew it: an empirical study of zero-day attacks in the real world. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012, pp. 833–844. ACM, New York (2012)
Brumley, D., Hartwig, C., Kang, M.G., Liang, Z., Newsome, J., Poosankam, P., Song, D., Yin, H.: Bitscope: automatically dissecting malicious binaries. Technical report, In CMU-CS-07-133 (2007)
Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., Yin, H.: Automatically identifying trigger-based behavior in malware. In: Lee, W., Wang, C., Dagon, D. (eds.) Botnet Detection. Advances in Information Security, vol. 36. Springer, Boston (2008). https://doi.org/10.1007/978-0-387-68768-1_4
Cadar, C., Dunbar, D., Engler, D.R.: Klee: unassisted and automatic generation of high-coverage tests for complex systems programs. In: OSDI, vol. 8, pp. 209–224 (2008)
Cha, S.K., Avgerinos, T., Rebert, A., Brumley, D.: Unleashing mayhem on binary code. In: Proceedings of the 2012 IEEE Symposium on Security and Privacy, SP 2012, pp. 380–394. IEEE Computer Society, Washington, DC (2012)
Chen, X., Andersen, J., Mao, Z.M., Bailey, M., Nazario, J.: Towards an understanding of anti-virtualization and anti-debugging behavior in modern malware. In: 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN), pp. 177–186, June 2008
Comparetti, P.M., Salvaneschi, G., Kirda, E., Kolbitsch, C., Kruegel, C., Zanero, S.: Identifying dormant functionality in malware programs. In: 2010 IEEE Symposium on Security and Privacy (SP), pp. 61–76. IEEE (2010)
Cuckoo: Automated malware analysis - cuckoo sandbox (2016). http://www.cuckoosandbox.org/
Dinaburg, A., Royal, P., Sharif, M., Lee, W.: Ether: malware analysis via hardware virtualization extensions. In: CCS 2008, pp. 51–62. ACM (2008)
Ferrie, T.L.: Win32.netsky.c. https://www.symantec.com/security_response/writeup.jsp?docid=2004-022417-4628-99
Fleck, D., Tokhtabayev, A., Alarif, A., Stavrou, A., Nykodym, T.: Pytrigger: a system to trigger & extract user-activated malware behavior. In: 2013 Eighth International Conference on Availability, Reliability and Security (ARES), pp. 92–101. IEEE (2013)
GeorgiaTech: Open malware (2016). http://www.offensivecomputing.net/
Gettis, S.: W32.mydoom.b@mm. https://www.symantec.com/security_response/writeup.jsp?docid=2004-022011-2447-99
Godefroid, P.: Compositional dynamic test generation. In: Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2007, pp. 47–54. ACM, New York (2007)
Google: Virustotal (2016). https://www.virustotal.com/
Graziano, M., Leita, C., Balzarotti, D.: Towards network containment in malware analysis systems. In: Proceedings of the 28th Annual Computer Security Applications Conference, ACSAC 2012, pp. 339–348. ACM, New York (2012)
Hindocha, N.: Win32.netsky.d. https://www.symantec.com/security_response/writeup.jsp?docid=2004-030110-0232-99
Kaspersky: Duqu (2016). http://www.kaspersky.com/about/press/major_malware_outbreaks/duqu
Kirat, D., Vigna, G., Kruegel, C.: Barecloud: bare-metal analysis-based evasive malware detection. In: Proceedings of the 23rd USENIX conference on Security Symposium (SEC 2014), pp. 287–301. USENIX Association, Berkeley (2014)
Kolbitsch, C., Comparetti, P.M., Kruegel, C., Kirda, E., Zhou, X., Wang, X.: Effective and efficient malware detection at the end host. In: Proceedings of the 18th Conference on USENIX Security Symposium, SSYM 2009, pp. 351–366. USENIX Association, Berkeley (2009)
Kolbitsch, C., Kirda, E., Kruegel, C.: The power of procrastination: detection and mitigation of execution-stalling malicious code (2011)
Kolbitsch, C., Livshits, B., Zorn, B., Seifert, C.: Rozzle: de-cloaking internet malware. In: Proceedings of the 2012 IEEE Symposium on Security and Privacy, SP 212, pp. 443–457. IEEE Computer Society, Washington, DC (2012)
Lindorfer, M., Kolbitsch, C., Milani Comparetti, P.: Detecting environment-sensitive malware. In: Sommer, R., Balzarotti, D., Maier, G. (eds.) RAID 2011. LNCS, vol. 6961, pp. 338–357. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23644-0_18
Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: IEEE Symposium on Security and Privacy, SP 2007, pp. 231–245 (2007)
Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: Twenty-Third Annual Computer Security Applications Conference, ACSAC 2007, pp. 421–430 (2007)
Nappa, A., Xu, Z., Rafique, M.Z., Caballero, J., Gu, G.: Cyberprobe: towards internet-scale active detection of malicious servers. In: Proceedings of the 2014 Network and Distributed System Security Symposium (NDSS 2014), pp. 1–15 (2014)
NetSky (2016). https://en.wikipedia.org/wiki/Netsky_(computer_worm)
Peng, F., Deng, Z., Zhang, X., Xu, D., Lin, Z., Su, Z.: X-force: force-executing binary programs for security applications. In: Proceedings of the 23rd USENIX Conference on Security Symposium, SEC 2014, pp. 829–844. USENIX Association, Berkeley (2014)
Porras, P., Saïdi, H., Yegneswaran, V.: A foray into conficker’s logic and rendezvous points. In: Proceedings of the 2nd USENIX Conference on Large-scale Exploits and Emergent Threats: Botnets, Spyware, Worms, and More, LEET 2009, p. 7. USENIX Association, Berkeley (2009)
Shin, S., Xu, Z., Gu, G.: Effort: efficient and effective bot malware detection. In: 2012 Proceedings IEEE INFOCOM, pp. 2846–2850, March 2012
Song, C., Royal, P., Lee, W.: Impeding automated malware analysis with environmentsensitive malware. In: USENIX Workshop on Hot Topics in Security (2012)
Symantec: Bifrost (2016). http://www.symantec.com/security_response/writeup.jsp?docid=2004-101214-5358-99
Symantec: Koobface (2016). http://www.symantec.com/security_response/writeup.jsp?docid=2008-080315-0217-99&tabid=2
Symantec: Sality (2016). http://www.symantec.com/security_response/writeup.jsp?docid=2006-011714-3948-99
Symantec: Symantec intelligence quarterly (2016). http://www.symantec.com/threatreport/quarterly.jsp
Symantec: Triage analysis of targeted attacks (2016). http://www.symantec.com/threatreport/topic.jsp?id=malicious_code_trend
Symantec: Trojan.neloweg (2016). http://www.symantec.com/security_response/writeup.jsp?docid=2012-020609-4221-99
Symantec: Zeus Trojan Horse (2016). http://www.symantec.com/security_response/writeup.jsp?docid=2010-011016-3514-99
UCSB: Angr (2016). https://github.com/angr/angr
Unicorn: The ultimate CPU emulator (2016). http://www.unicorn-engine.org/
Wikipedia: Flame (2016). http://en.wikipedia.org/wiki/Flame_malware
Wikipedia: Stuxnet (2016). http://en.wikipedia.org/wiki/Stuxnet
Wikipedia: Trojan backdoor.flashback (2016). http://en.wikipedia.org/wiki/Trojan_BackDoor.Flashback
Wilhelm, J., Chiueh, T.C.: A forced sampled execution approach to kernel rootkit identification (2007)
Willems, C., Holz, T., Freiling, F.: Toward automated dynamic malware analysis using cwsandbox. IEEE Secur. Privacy 5(2), 32–39 (2007)
Xu, Z., Zhang, J., Gu, G., Lin, Z.: Autovac: automatically extracting system resource constraints and generating vaccines for malware immunization. In: 2013 IEEE 33rd International Conference on Distributed Computing Systems (ICDCS), pp. 112–123, July 2013
Xu, Z., Chen, L., Gu, G., Kruegel, C.: Peerpress: utilizing enemies’ P2P strength against them. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 212, pp. 581–592. ACM, New York (2012)
Xu, Z., Zhang, J., Gu, G., Lin, Z.: GoldenEye: efficiently and effectively unveiling malware’s targeted environment. In: Stavrou, A., Bos, H., Portokalidis, G. (eds.) RAID 2014. LNCS, vol. 8688, pp. 22–45. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11379-1_2
Acknowledgments
This work was partially supported by The National Key Research and Development Program of China (2016YFB0801004 and 2016YFB0801604).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Li, Q., Zhang, Y., Su, L., Wu, Y., Ma, X., Yang, Z. (2018). An Improved Method to Unveil Malware’s Hidden Behavior. In: Chen, X., Lin, D., Yung, M. (eds) Information Security and Cryptology. Inscrypt 2017. Lecture Notes in Computer Science(), vol 10726. Springer, Cham. https://doi.org/10.1007/978-3-319-75160-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-75160-3_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-75159-7
Online ISBN: 978-3-319-75160-3
eBook Packages: Computer ScienceComputer Science (R0)