Skip to main content
SpringerLink
Log in

Joint network-host based malware detection using information-theoretic tools

  • Original Paper
  • Published:
Journal in Computer Virology Aims and scope Submit manuscript

Abstract

In this paper, we propose two joint network-host based anomaly detection techniques that detect self-propagating malware in real-time by observing deviations from a behavioral model derived from a benign data profile. The proposed malware detection techniques employ perturbations in the distribution of keystrokes that are used to initiate network sessions. We show that the keystrokes’ entropy increases and the session-keystroke mutual information decreases when an endpoint is compromised by a self-propagating malware. These two types of perturbations are used for real-time malware detection. The proposed malware detection techniques are further compared with three prominent anomaly detectors, namely the maximum entropy detector, the rate limiting detector and the credit-based threshold random walk detector. We show that the proposed detectors provide considerably higher accuracy with almost 100% detection rates and very low false alarm rates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Khayam, S.A., Radha, H.: Using session-keystroke mutual information to detect self-propagating malicious codes. In: IEEE ICC, June 2007

  2. Ellis, D., Aiken, J.G., Attwood, K.S., Tenaglia, S.D.: A behavioral approach to worm detection. In: ACM Workshop on Rapid Malcode (WORM), October 2004

  3. Zou, C.C., Gao, L., Gong, W., Towsley, D.: Monitoring and early warning of Internet worms. In: ACM Conference on Computer and Communications Security (CCS), October 2003

  4. Wu, J., Vangala, S., Gao, L.: An effective architecture and algorithm for detecting worms with various scan techniques. In: Network and Distributed System Security Symposium (NDSS), February 2004

  5. Jung, J., Paxson, V., Berger, A.W., Balakrishnan, H.: Fast portscan detection using sequential hypothesis testing. In: IEEE Oakland Symposium on Security and Privacy, May 2004

  6. Weaver, N., Staniford, S., Paxson, V.: Very fast containment of scanning worms. In: Usenix Security Symposium, August 2004

  7. Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: ACM SIGCOMM, August/September 2004

  8. Lakhina, A., Crovella, M., Diot, C.: Characterization of network-wide traffic anomalies in traffic flows. In: ACM Internet Measurement Conference (IMC), October 2004

  9. Barford, P., Kline, J., Plonka, D., Ron, A.: A signal analysis of network traffic anomalies. In: ACM Internet Measurement Conference (IMC), November 2002

  10. Krishnamurthy, B., Sen, S., Zhang, Y., Chen, Y.: Sketch-based change detection: methods, evaluation, and applications. ACM Internet Measurement Conference (IMC), October 2003

  11. Soule, A., Salamatian, K., Taft, N.: Combining filtering and statistical methods for anomaly detection. In: ACM/Usenix Internet Measurement Conference (IMC), October 2005

  12. Kim, Y., Lau, W.C., Chuah, M.C., Chao, H.J.: PacketScore: statistics-based overload control against distributed denial-of-service attacks. In: IEEE INFOCOM, March 2004

  13. Lakhina, A., Crovella, M., Diot, C.: Mining anomalies using traffic feature distributions. In: ACM SIGCOMM, August 2005

  14. Gu, Y., McCullum, A., Towsley, D.: Detecting anomalies in network traffic using maximum entropy estimation. In: ACM/Usenix Internet Measurement Conference (IMC), October 2005

  15. Moore, D., Shannon, C., Voelker, G.M., Savage, S.: Network telescopes. CAIDA technical report. http://www.caida.org/outreach/papers/2004/tr-2004-04/

  16. Cooke, E., Bailey, M., Mao, Z.M., Watson, D., Jahanian, F., McPherson, D.: Toward understanding distributed blackhole placement. In: ACM Workshop on Rapid Malcode (WORM), October 2004

  17. Bailey, M., Cooke, E., Jahanian, F., Nazario, J., Watson, D.: The Internet motion sensor: a distributed blackhole monitoring system. In: Network and Distributed System Security Symposium (NDSS), February 2005

  18. Dagon, D., Qin, X., Gu, G., Lee, W.: HoneyStat: local worm detection using honeypots. In: International Symposium on Recent Advances in Intrusion Detection (RAID), September 2004

  19. Twycross, J., Williamson, M.M.: Implementing and testing a virus throttle. In: Usenix Security Symposium, August 2003

  20. Schechter, S.E., Jung, J., Berger, A.W.: Fast detection of scanning worm infections. In: RAID (2004)

  21. Williamson, M.M.: Throttling viruses: restricting propagation to defeat malicious mobile code. In: Annual Computer Security Applications Conference (ACSAC), December 2002

  22. Sellke, S., Shroff, N.B., Bagchi, S.: Modeling and automated containment of worms. In: International Conference on Dependable Systems and Networks (DSN), June/July 2005

  23. Whyte, D., Kranakis, E., van Oorschot, P.C.: DNS-based detection of scanning worms in an enterprise network. In: Network and Distributed System Security Symposium (NDSS), February 2005

  24. Gupta, A., Sekar, R.: An approach for detecting self-propagating email using anomaly detection. In: International Symposium on Recent Advances in Intrusion Detection (RAID), September 2003

  25. Xiong, J.: ACT: attachment chain tracing scheme for email virus detection and control. In: ACM Workshop on Rapid Malcode (WORM), October 2004

  26. Me, L., Michel, C.: Intrusion detection: a bibliography. Tech. Rep. SSIR-2001-01, September 2001

  27. Cui, W., Katz, R.H., Tan, W.-T.: BINDER: an extrusion-based break-in detector for personal computers. In: Usenix Security Symposium, April 2005

  28. Ilgun K., Kemmerer R.A., Porras P.A.: State transition analysis: a rule-based intrusion detection approach. IEEE Trans. Softw. Eng. 21(3), 181–199 (1995)

    Article  Google Scholar 

  29. Jha, S., Tan, K., Maxion, R.A.: Markov Chains, classifiers, and intrusion detection. In: IEEE CSFW, June 2001

  30. Ye, N.: A Markov Chain model of temporal behavior for anomaly detection. In: IEEE Workshop on Information Assurance and Security, June 2000

  31. DuMouchel, W.: Computer intrusion detection based on bayes factors for comparing command transition probabilities. Tech. Rep. 91, National Institute of Statistical Sciences (1999)

  32. Lazarevic, A., Ozgur, A., Ertoz, L., Srivastava, J., Kumar, V.: A comparative study of anomaly detection schemes in network intrusion detection. In: SIAM Conference on Data Mining, May 2003

  33. Lippmann, R.P., et al.: The 1998 DARPA/AFRL off-line intrusion detection evaluation. In: RAID, September 1998

  34. Lippmann R.P., Haines J.W., Fried D.J., Korba J., Das K.: The 1999 DARPA off-line intrusion detection evaluation. ACM Comput Netw 34(4), 579–595 (2000)

    Article  Google Scholar 

  35. Endpoint Security Homepage. http://www.endpointsecurity.org/

  36. Symantec Internet Security Threat Report XI. Trends for July–December 07. March 2007

  37. Raschke, T.: The new security challenge: endpoints. IDC/F-Secure, August 2005

  38. Weaver, N., Ellis, D., Staniford, S., Paxson, V.: Worms vs. perimeters: the case for hard-LANs. In: IEEE Symposium on High Performance Interconnects (Hot Interconnects), August 2004

  39. Wong, C., Wang, C., Song, D., Bielski, S., Ganger, G.R.: Dynamic quarantine of Internet worms. In: International Conference on Dependable Systems and Networks (DSN), July 2004

  40. Wong, C., Bielski, S., Studer, A., Wang, C.: Empirical analysis of rate limiting mechanisms. In: International Symposium on Recent Advances in Intrusion Detection (RAID), September 2005

  41. Li, Q., Chang, E.-C., Chan, M.C.: On effectiveness of DDOS attacks on statistical filtering. IEEE Infocom, March 2005

  42. Kuzmanovic, A., Knightly, E.W.: Low-rate TCP-targeted denial of service attacks. In: ACM SIGCOMM, August 2003

  43. Staniford, S., Paxson, V., Weaver, N.: How to own the Internet in your spare time. In: Usenix Security Symposium, August 2002

  44. Panjwani, S., Tan, S., Jarrin, K.M., Cukier, M.: An experimental evaluation to determine if port scans are precursor to an attack. In: International Conference on Dependable Systems and Networks (DSN), June/July 2005

  45. Cover T.M., Thomas J.A.: Elements of Information Theory. Wiley-Interscience, New York (1991)

    Book  MATH  Google Scholar 

  46. SHA-1. The Secure Hash Algorithm. FIPS PUB 180-1, April 1995

  47. MSDN Library. http://msdn.microsft.com

  48. Microsoft Virtual PC 2004. http://www.microsoft.com/Windows/virtualpc

  49. Sophos Virus Info. http://www.sophos.com/virusinfo/

  50. Symantec Security Response. http://securityresponse.symantec.com/avcenter

  51. TrendMicro Virus Encyclopedia. http://au.trendmicro-europe.com/smb/vinfo

  52. Kumar, A., Paxson, V., Weaver, N.: Exploiting underlying structure for detailed reconstruction of an Internet-scale event. In: ACM/ Usenix Internet Measurement Conference (IMC), October 2005

  53. Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: ACM CCS, November 2002

  54. Trusted Computing Alliance. https://www.trustedcomputinggroup.org

  55. Dunlap, G., King, S., Cinar, S., Basrai, M., Chen, P.: ReVirt: enabling intrusion analysis through virtual-machine logging and replay. Usenix OSDI, December 2002

  56. Garfinkel, T., Pfaff, B., Chow, J., Rosenblum, M., Boneh, D.: Terra: a virtual machine-based platform for trusted computing. ACM SOSP, October 2003

  57. Lampson B.W.: Computer security in the real world. IEEE Comput. 37(6), 37–46 (2004)

    Google Scholar 

  58. Rosenblum M., Garfinkel T.: Virtual machine monitors: current technology and future trends. IEEE Comput. 38(5), 39–47 (2005)

    Google Scholar 

  59. Wong, C., Bielski, S., Studer, A., Wang, C.: Empirical analysis of rate limiting mechanisms. In: RAID (2005)

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST), Islamabad, Pakistan

    Syed Ali Khayam & Ayesha Binte Ashfaq

  2. Department of Electrical and Computer Engineering, Michigan State University (MSU), East Lansing, USA

    Hayder Radha

Authors
  1. Syed Ali Khayam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayesha Binte Ashfaq
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hayder Radha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Syed Ali Khayam.

Additional information

Parts of this work appeared in the Proceedings of IEEE International Conference on Communications (ICC) 2007 [1].

S. A. Khayam’s work was supported in part by Pakistan National ICT R&D Fund and Higher Education Commission (HEC), Pakistan. H. Radha’s work was supported in part by NSF Award CNS-0430436, NSF Award CCF-0515253, MEDC Grant GR-296, and an unrestricted gift from Microsoft Research.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Khayam, S.A., Ashfaq, A.B. & Radha, H. Joint network-host based malware detection using information-theoretic tools. J Comput Virol 7, 159–172 (2011). https://doi.org/10.1007/s11416-010-0145-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11416-010-0145-1

Keywords

Search

Navigation