Abstract
Previous research has applied classic cryptanalytic techniques to the malware detection problem. Specifically, scores that are based on simple substitution cipher cryptanalysis have been considered. In this research, we analyze two malware scoring techniques based on the classic Vigenère cipher. Our first approach relies only on the index of coincidence (IC), which is used for example, to determine the length of the keyword in a Vigenère ciphertext. To compute the IC, we consider both the Kasisky Test and Friedman’s Test. We also consider a score based on a more complete cryptanalysis of a Vigenère cipher, where the IC calculation is the first step. We find that both of these scores outperform comparable malware scores in selected cases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Annachatre, C., Austin, T.H., Stamp, M.: Hidden Markov models for malware classification. J. Comput. Virol. Hacking Tech. 11(2), 59–73 (2015)
Bradley, A.P.: The use of the area under the ROC curve in the evolution of machine learning algorithms. Pattern Recogn. 30, 1145–1159 (1997)
Cryptanalysis of Vigenère Cipher and Substitution Cipher. http://shodhganga.inflibnet.ac.in/bitstream/10603/26543/10/10_chapter5.pdf
Deshmukh, S.: Vigenère Cipher for Malware Detection, Master’s Report, Department of Computer Science, San Jose State University (2016)
F-Secure Annual Security Report 2007. https://www.f-secure.com/documents/10192/1118990/AnnualReport_2007_en.pdf/
Fawcett, T.: An introduction to ROC analysis. Pattern Recognit. Lett. 27(8), 861–874 (2006)
Friedman, W.F.: The Index of Coincidence and Its Applications in Cryptography. Aegean Park Press, Walnut Creek (1987)
Gilleland, M.: Levenshtein distance in three flavors. http://people.cs.pitt.edu/~kirk/cs1501/Pruhs/Spring2006/assignments/editdistance/Levenshtein
Harebot, M: Panda Security. http://www.pandasecurity.com/usa/homeusers/security-info/220319/Harebot.M/
Jakobsen, T.: A fast method for the cryptanalysis of substitution ciphers. Cryptologia 19, 265–274 (1995)
Kaspersky Lab. http://support.kaspersky.com/viruses/rogue?qid=208286454
Lin, D., Stamp, M.: Hunting for undetectable metamorphic viruses. J. Comput. Virol. 7(3), 201–214 (2011)
Malicia Project, 2015. http://malicia-project.com/
Nappa, A., Zubair Rafique, M., Caballero, J.: Driving in the cloud: an analysis of drive-by download operations and abuse reporting. In: Proceedings of the 10th Conference on Detection of Intrusions and Malware and Vulnerability Assessment. Berlin, Germany, (July 2013)
Next Generation Virus Construction Kit (NGVCK). http://vxheaven.org/vx.php?id=tn02
Runwal, N., Low, R., Stamp, M.: Opcode graph similarity and metamorphic detection. J. Comput. Virol. Hacking Tech. 8(2), 37–52 (2012)
SecurityShield, Microsoft Malware Protection Center. http://www.microsoft.com/security/portal/threat/encyclopedia/Entry.aspx?Name=SecurityShield
Shanmugam, G., Low, R.M., Stamp, M.: Simple substitution distance and metamorphic detection. J. Comput. Virol. Hacking Tech. 9(3), 159–170 (2013)
S, Sing: The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography. Anchor, New York (2011)
Smart HDD, 2015. http://support.kaspersky.com/viruses/rogue?qid=208286454
Srinivasan, S.: SSCT Score for Malware Detection, Master’s report, Department of Computer Science, San Jose State University (2015). http://scholarworks.sjsu.edu/etd_projects/444/
Stamp, M., Low, R .M.: Applied Cryptanalysis: Breaking Ciphers in the Real World. Wiley, Hoboken (2006)
Stamp, M.: Information Security: Principles and Practice, 2nd edn. Wiley, Hoboken (2011)
Stamp, M.: Machine Learning with Applications in Information Security. Chapman and Hall/CRC, Boca Raton (2017)
Symantec Annual Security Report 2008. http://www.realwire.com/releases/symantec-announces-messagelabs-intelligence-2008-annual-security-report
Szor, P.: The Art of Computer Virus Research and Defense. Pearson Eduction, Upper Saddle River (2005)
Trojan.Cridex, Symantec, 2012. http://www.symantec.com/security_response/writeup.jsp?docid=2012-012103-0840-99
Trojan.Zbot,, Symantec, 2010. http://www.symantec.com/security_response/writeup.jsp?docid=2010-011016-3514-99
Trojan.ZeroAccess, Symantec, 2013. http://www.symantec.com/security_response/writeup.jsp?docid=2011-071314-0410-99
Vigenère and Gronsfeld Cipher, Practical Cryptography. http://practicalcryptography.com/ciphers/vigenere-gronsfeld-and-autokey-cipher/
Vigenère Cipher, Crypto Museum. http://www.cryptomuseum.com/crypto/vigenere/
Wong, W., Stamp, M.: Hunting for metamorphic engines. J. Comput. Virol. 2(3), 211–229 (2006)
Winwebsec, Microsoft Malware Protection Center. https://www.microsoft.com/security/portal/threat/encyclopedia/entry.aspx?Name=Win32
Yi, J.: Cryptanalysis of Homophonic Substitution-Transposition Cipher, Department of Computer Science, San Jose State University (2014) http://scholarworks.sjsu.edu/etd_projects/357/
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Deshmukh, S., Troia, F.D. & Stamp, M. Vigenère scores for malware detection. J Comput Virol Hack Tech 14, 157–165 (2018). https://doi.org/10.1007/s11416-017-0300-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11416-017-0300-z