skip to main content
10.1145/2851613.2851802acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

MalFlow: identification of C&C servers through host-based data flow profiling

Published:04 April 2016Publication History

ABSTRACT

Modern malware interacts with multiple internet domains for various reasons: communication with command and control (C&C) servers, boosting click counts on online ads or performing denial of service attacks, among others. The identification of malign domains is thus necessary to prevent (and react to) incidents. Since malware creators constantly generate new domains to avoid detection, maintaining up-to-date lists of malign domains is challenging. We propose an approach that automatically estimates the risk associated with communicating with a domain based on the data flow behavior of a process communicating with it. Our approach uses unsupervised learning on data flow profiles that capture communication of processes with network endpoints at system call level to distinguish between likely malign or benign behavior. Our evaluations on a large and diverse data set indicate a high detection accuracy and a reasonable performance overhead. We further discuss how this concept can be used in an operational setting for fine-grained enforcement of risk-based incident response actions.

References

  1. Bilge, L., Balzarotti, D., Robertson, W., Kirda, E., Kruegel, C.: Disclosure: Detecting botnet command and control servers through large-scale netflow analysis. In: ACSAC (2012) Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bilge, L., Kirda, E., Kruegel, C., Balduzzi, M.: Exposure: Finding malicious domains using passive dns analysis. In: NDSS (2011)Google ScholarGoogle Scholar
  3. Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., Yin, H.: Automatically identifying trigger-based behavior in malware. In: Botnet Detection (2008)Google ScholarGoogle Scholar
  4. Caballero, J., Venkataraman, S., Poosankam, P., Kang, M. G., Song, D., Blum, A.: Fig: Automatic fingerprint generation. CMU TechReport (2007)Google ScholarGoogle Scholar
  5. Egele, M., Scholte, T., Kirda, E., Kruegel, C.: A survey on automated dynamic malware-analysis techniques and tools. CSUR (2012) Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Gu, G., Perdisci, R., Zhang, J., Lee, W., et al.: Botminer: Clustering analysis of network traffic for protocol-and structure-independent botnet detection. In: USENIX Sec (2008) Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Gu, G., Zhang, J., Lee, W.: Botsniffer: Detecting botnet command and control channels in network traffic. In: NDSS (2008)Google ScholarGoogle Scholar
  8. Jacob, G., Hund, R., Kruegel, C., Holz, T.: Jackstraws: Picking command and control connections from bot traffic. In: USENIX Sec (2011) Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kheir, N., Blanc, G., Debar, H., Garcia-Alfaro, J., Yang, D.: Automated classification of c&c connections through malware url clustering. In: SEC (2015)Google ScholarGoogle Scholar
  10. Kührer, M., Rossow, C., Holz, T.: Paint it black: Evaluating the effectiveness of malware blacklists. In: RAID (2014)Google ScholarGoogle Scholar
  11. Nappa, A., Xu, Z., Rafique, M. Z., Caballero, J., Gu, G.: Cyberprobe: Towards internet-scale active detection of malicious servers. In: NDSS (2014)Google ScholarGoogle Scholar
  12. Oktavianto, D., Muhardianto, I.: Cuckoo Malware Analysis. Packt Pbl. Ltd (2013) Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Rafique, M. Z., Caballero, J.: Firma: Malware clustering and network signature generation with mixed network behaviors. In: RAID (2013) Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Sculley, D.: Web-scale k-means clustering. In: WWW (2010) Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Wressnegger, C., Schwenk, G., Arp, D., Rieck, K.: A close look on n-grams in intrusion detection: anomaly detection vs. classification. In: AISec (2013) Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Wüchner, T., Ochoa, M., Pretschner, A.: Malware detection with quantitative data flow graphs. In: ASIACCS (2014)Google ScholarGoogle Scholar
  17. Wüchner, T., Ochoa, M., Pretschner, A.: Robust and effective malware detection through quantitative data flow graph metrics. In: DIMVA (2015)Google ScholarGoogle Scholar
  18. Wüchner, T., Pretschner, A.: Data loss prevention based on data-driven usage control. In: ISSRE (2012)Google ScholarGoogle Scholar
  19. Wurzinger, P., Bilge, L., Holz, T., Goebel, J., Kruegel, C., Kirda, E.: Automatically generating models for botnet detection. In: ESORICS (2009) Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Xu, Z., Nappa, A., Baykov, R., Yang, G., Caballero, J., Gu, G.: Autoprobe: Towards automatic active malicious server probing using dynamic binary analysis. In: CCS (2014) Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Yadav, S., Reddy, A. K. K., Reddy, A., Ranjan, S.: Detecting algorithmically generated malicious domain names. In: IMC (2010) Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. You, I., Yim, K.: Malware obfuscation techniques: A brief survey. In: BWCAA (2010) Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. MalFlow: identification of C&C servers through host-based data flow profiling

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      SAC '16: Proceedings of the 31st Annual ACM Symposium on Applied Computing
      April 2016
      2360 pages
      ISBN:9781450337397
      DOI:10.1145/2851613

      Copyright © 2016 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 4 April 2016

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      SAC '16 Paper Acceptance Rate252of1,047submissions,24%Overall Acceptance Rate1,650of6,669submissions,25%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader