Abstract
Encrypted code is often present in some types of advanced malware, while such code virtually never appears in legitimate applications. Hence, the presence of encrypted code within an executable file could serve as a strong heuristic for malware detection. In this chapter, we consider the feasibility of detecting encrypted segments within an executable file using hidden Markov models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
An HMM score is dependent on the length of the sequence scored. Therefore, in each case we normalize the score so that it is given as a log likelihood per opcode (LLPO).
References
Swain B (2009) What are malware, viruses, spyware, and cookies, and what differentiates them? https://www.symantec.com/connect/articles/what-are-malware-viruses-spyware-and-cookies-and-what-differentiates-them
Nachenberg C (1996) Understanding and managing polymorphic viruses. In: The symantec enterprise papers. Symantec. https://www.symantec.com/content/dam/symantec/docs/security-center/white-papers/understanding-and-managing-polymorphic-viruses-96-en.pdf
Computer Knowledge (2013). http://www.cknow.com/cms/vtutor/types-of-viruses.html
Stamp M (2011) Information security: principles and practice. Wiley, New York
DaBoss (2013) Robert slade computer virus history. http://www.cknow.com/cms/vtutor/robert-slade-computer-virus-history.html
Radeska T (2016) Brain — The first computer virus, the vintage news. http://www.thevintagenews.com/2016/09/08/priority-brain-first-computer-virus-created-two-brothers-pakistan-just-wanted-prevent-customers-making-illegal-software-copies/
Szor P (2005) The art of computer virus research and defense. Pearson Education. https://books.google.com/books?id=XE-ddYF6uhYC
Rad BB, Masrom M, Ibrahim S (2012) Camouflage in malware: from encryption to metamorphism. Int J Comput Sci Netw Secur 12(8):74–83
Li X, Loh PKK, Tan F (2011) Mechanisms of polymorphic and metamorphic viruses. In: 2011 European intelligence and security informatics conference. pp 149–154
Symantec: viruses that can cost you. http://www.symantec.com/region/reg_eu/resources/virus_cost.html
Symantec: security 1:1 — Part 1: viruses and worms (2013). https://www.symantec.com/connect/articles/security-11-part-1-viruses-and-worms
Venkatachalam S (2010) Detecting undetectable computer viruses. http://scholarworks.sjsu.edu/etd_projects/156/
Zwanger V, Gerhards-Padilla E, Meier M (2014) Codescanner: Detecting (hidden) x86/x64 code in arbitrary files. In: Malicious and unwanted software: the americas (MALWARE), 2014 9th international conference on malicious and unwanted software. IEEE, pp 118–127
Schultz MG, Eskin E, Zadok F, Stolfo SJ (2001) Data mining methods for detection of new malicious executables. In: Proceedings of the IEEE symposium on security and privacy. SP, pp 38–49
Shabtai A, Moskovitch R, Elovici Y, Glezer C (2009) Detection of malicious code by applying machine learning classifiers on static features: a state-of-the-art survey. Inf Secur Tech Rep 14(1):16–29. https://doi.org/10.1016/j.istr.2009.03.003
Stamp M (2004) A revealing introduction to hidden Markov models. https://www.cs.sjsu.edu/~stamp/RUA/HMM.pdf
Jurafsky D, Martin JH (2000) Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 1st edn. Prentice Hall PTR, USA
Cave RL, Neuwirth LP (1980) Hidden Markov models for English. In: Ferguson JD (ed) Hidden Markov Models for Speech
Brown corpus of standard American English (2010). http://www.cs.toronto.edu/~gpenn/csc401/a1res.html
Vobbilisetty R, Troia FD, Low RM, Visaggio CA, Stamp M (2017) Classic cryptanalysis using hidden Markov models. Cryptologia 41(1):1–28. https://doi.org/10.1080/01611194.2015.1126660
Ganesh N, Di Troia F, Corrado VA, Austin TH, Stamp M (2016) Static analysis of malicious Java applets. In: Proceedings of the 2016 ACM on international workshop on security and privacy analytics. IWSPA ’16. ACM, USA, pp 58–63, http://doi.acm.org/10.1145/2875475.2875477
Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. IEEE Proc 77(2):257–286
Shanmugam G, Low RM, Stamp M (2013) Simple substitution distance and metamorphic detection. J Comput Virol Hacking Tech 9(3):159–170
Wong W, Stamp M (2006) Hunting for metamorphic engines. J Comput Virol 2(3):211–229. https://doi.org/10.1007/s11416-006-0028-7
Shamir A, Van Someren N (1999) Playing hide and seek with stored keys. In: International conference on financial cryptography. Springer, Berlin, pp 118–124
Dhanasekar D (2017) Detecting encrypted malware using hidden Markov models. Master’s project, Department of Computer Science, San Jose State University. http://scholarworks.sjsu.edu/etd_projects/574/
Bradley AP (1997) The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognit 30(7):1145–1159
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Dhanasekar, D., Di Troia, F., Potika, K., Stamp, M. (2018). Detecting Encrypted and Polymorphic Malware Using Hidden Markov Models. In: Parkinson, S., Crampton, A., Hill, R. (eds) Guide to Vulnerability Analysis for Computer Networks and Systems. Computer Communications and Networks. Springer, Cham. https://doi.org/10.1007/978-3-319-92624-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-92624-7_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92623-0
Online ISBN: 978-3-319-92624-7
eBook Packages: Computer ScienceComputer Science (R0)