research-article

A Graph-based Model for Malicious Software Detection Exploiting Domination Relations between System-call Groups

Authors:

Stavros D. Nikolopoulos,

Iosif PolenakisAuthors Info & Claims

CompSysTech '18: Proceedings of the 19th International Conference on Computer Systems and Technologies

Pages 20 - 26

https://doi.org/10.1145/3274005.3274028

Published: 13 September 2018 Publication History

Abstract

In this paper, we propose a graph-based algorithmic technique for malware detection, utilizing the System-call Dependency Graphs (ScDG) obtained through taint analysis traces. We leverage the grouping of system-calls into system-call groups with respect to their functionality to merge disjoint vertices of ScDG graphs, transforming them to Group Relation Graphs (GrG); note that, the GrG graphs represent malware's behavior being hence more resilient to probable mutations of its structure. More precisely, we extend the use of GrG graphs by mapping their vertices on the plane utilizing the degrees and the vertex-weights of a specific underlying graph of the GrG graph as to compute domination relations. Furthermore, we investigate how the activity of each system-call group could be utilized in order to distinguish graph-representations of malware and benign software. The domination relations among the vertices of GrG graphs result to a new graph representation that we call Coverage Graph of the GrG graph. Finally, we evaluate the potentials of our detection model using graph similarity between Coverage Graphs of known malicious and benign software samples of various types.

References

[1]

Manoun Alazab, Robert Layton, Sitalakshmi Venkataraman, and Paul Watters. Malware detection based on structural and behavioural features of api calls. (2010).

[2]

Domagoj Babić, Daniel Reynaud, and Dawn Song. 2011. Malware analysis with tree automata inference. In International Conference on Computer Aided Verification. Springer, 116--131.

Digital Library

[3]

Mihai Christodorescu, Somesh Jha, and Christopher Kruegel. 2007. Mining specifications of malicious behavior. In Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering. ACM, 5--14.

Digital Library

[4]

Mihai Christodorescu, Somesh Jha, Sanjit A Seshia, Dawn Song, and Randal E Bryant. 2005. Semantics-aware malware detection. In Security and Privacy, 2005 IEEE Symposium on. IEEE, 32--46.

Digital Library

[5]

Yuxin Ding, Xiaoling Xia, Sheng Chen, and Ye Li. 2018. A malware detection method based on family behavior graph. Computers & Security 73 (2018), 73--86.

[6]

Matt Fredrikson, Somesh Jha, Mihai Christodorescu, Reiner Sailer, and Xifeng Yan. 2010. Synthesizing near-optimal malware specifications from suspicious behaviors. In Security and Privacy (SP), 2010 IEEE Symposium on. IEEE, 45--60.

Digital Library

[7]

Mehadi Hassen and Philip K Chan. 2017. Scalable function call graph-based malware classification. In Proceedings of the Seventh ACM on Conference on Data and Application Security and Privacy. ACM, 239--248.

Digital Library

[8]

Clemens Kolbitsch, Paolo Milani Comparetti, Christopher Kruegel, Engin Kirda, Xiao-yong Zhou, and XiaoFeng Wang. 2009. Effective and Efficient Malware Detection at the End Host. In USENIX security symposium, Vol. 4. 351--366.

Digital Library

[9]

Aziz Makandar and Anita Patrot. 2018. Trojan Malware Image Pattern Classification. In Proceedings of International Conference on Cognition and Recognition. Springer, 253--262.

[10]

Kirti Mathur and Saroj Hiranwal. 2013. A survey on techniques in detection and analyzing malware executables. International Journal of Advanced Research in Computer Science and Software Engineering 3, 4 (2013).

[11]

Sharma Divya Mukesh, Jigar A Raval, and Hardik Upadhyay. 2017. Real-Time Framework for Malware Detection Using Machine Learning Technique. In International Conference on Information and Communication Technology for Intelligent Systems. Springer, 173--182.

[12]

Stavros D Nikolopoulos and Iosif Polenakis. 2017. A graph-based model for malware detection and classification using system-call groups. Journal of Computer Virology and Hacking Techniques 13, 1 (2017), 29--46.

[13]

Younghee Park, Douglas Reeves, Vikram Mulukutla, and Balaji Sundaravel. 2010. Fast malware classification by automated behavioral graph matching. In Proceedings of the Sixth Annual Workshop on Cyber Security and Information Intelligence Research. ACM, 45.

Digital Library

[14]

Babak Bashari Rad, Maslin Masrom, and Suhaimi Ibrahim. 2012. Camouflage in malware: from encryption to metamorphism. International Journal of Computer Science and Network Security 12, 8 (2012), 74--83.

[15]

Lubomir Sikora and Ivan Zelinka. 2018. Swarm Virus, Evolution, Behavior and Networking. In Evolutionary Algorithms, Swarm Dynamics and Complex Networks. Springer, 213--239.

[16]

Michael Sikorski and Andrew Honig. 2012. Practical malware analysis: the handson guide to dissecting malicious software. no starch press.

Digital Library

[17]

Alireza Souri and Rahil Hosseini. 2018. A state-of-the-art survey of malware detection approaches using data mining techniques. Human-centric Computing and Information Sciences 8, 1 (2018), 3.

Digital Library

Cited By

Chysi ANikolopoulos SPolenakis I(2022)Detection and classification of malicious software utilizing Max-Flows between system-call groupsJournal of Computer Virology and Hacking Techniques10.1007/s11416-022-00433-219:1(97-123)Online publication date: 14-Jun-2022
https://doi.org/10.1007/s11416-022-00433-2
Nikolopoulos SPolenakis I(2022)Behavior-based detection and classification of malicious software utilizing structural characteristics of group sequence graphsJournal of Computer Virology and Hacking Techniques10.1007/s11416-022-00423-418:4(383-406)Online publication date: 15-Jun-2022
https://doi.org/10.1007/s11416-022-00423-4
Alaeiyan MParsa S(2022)A hierarchical layer of atomic behavior for malicious behaviors predictionJournal of Computer Virology and Hacking Techniques10.1007/s11416-022-00422-518:4(367-382)Online publication date: 7-Apr-2022
https://doi.org/10.1007/s11416-022-00422-5
Show More Cited By

Index Terms

A Graph-based Model for Malicious Software Detection Exploiting Domination Relations between System-call Groups

Recommendations

Malicious Software Detection utilizing Temporal-Graphs
CompSysTech '19: Proceedings of the 20th International Conference on Computer Systems and Technologies

In this work we propose a graph-based model that, utilizing relations between groups of System-calls, distinguishes malicious from benign software samples utilizing a behavioral graph representing their interaction with the operating system. More ...
A graph-based model for malicious code detection exploiting dependencies of system-call groups
CompSysTech '15: Proceedings of the 16th International Conference on Computer Systems and Technologies

In this paper, we propose a graph-based algorithmic technique for malware detection. More precisely, we utilize the system-call dependency graphs (or, for short, ScD graphs), obtained by capturing taint analysis traces and a set of various similarity ...
Malicious software classification based on relations of system-call groups
PCI '15: Proceedings of the 19th Panhellenic Conference on Informatics

In this paper we present a graph-based algorithmic technique for classifying unknown malware samples. In order for our model to be resistant against strong mutation of malicious software, we apply our classification technique on a weighted directed ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CompSysTech '18: Proceedings of the 19th International Conference on Computer Systems and Technologies

September 2018

206 pages

ISBN:9781450364256

DOI:10.1145/3274005

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

ERSVB: EURORISC SYSTEMS - Varna, Bulgaria
FOSEUB: FEDERATION OF THE SCIENTIFIC ENGINEERING UNIONS - Bulgaria
UORB: University of Ruse, Bulgaria
TECHUVB: Technical University of Varna, Bulgaria

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CompSysTech'18

CompSysTech'18: 19th International Conference on Computer Systems and Technologies

September 13 - 14, 2018

Ruse, Bulgaria

Acceptance Rates

Overall Acceptance Rate 241 of 492 submissions, 49%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
141
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)3

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chysi ANikolopoulos SPolenakis I(2022)Detection and classification of malicious software utilizing Max-Flows between system-call groupsJournal of Computer Virology and Hacking Techniques10.1007/s11416-022-00433-219:1(97-123)Online publication date: 14-Jun-2022
https://doi.org/10.1007/s11416-022-00433-2
Nikolopoulos SPolenakis I(2022)Behavior-based detection and classification of malicious software utilizing structural characteristics of group sequence graphsJournal of Computer Virology and Hacking Techniques10.1007/s11416-022-00423-418:4(383-406)Online publication date: 15-Jun-2022
https://doi.org/10.1007/s11416-022-00423-4
Alaeiyan MParsa S(2022)A hierarchical layer of atomic behavior for malicious behaviors predictionJournal of Computer Virology and Hacking Techniques10.1007/s11416-022-00422-518:4(367-382)Online publication date: 7-Apr-2022
https://doi.org/10.1007/s11416-022-00422-5
Dounavi HMpanti ANikolopoulos SPolenakis I(2021)A graph-based framework for malicious software detection and classification utilizing temporal-graphsJournal of Computer Security10.3233/JCS-210057(1-38)Online publication date: 27-Aug-2021
https://doi.org/10.3233/JCS-210057
Dounavi HMpanti ANikolopoulos SPolenakis I(2021)Detection and Classification of Malicious Software based on Regional Matching of Temporal GraphsProceedings of the 22nd International Conference on Computer Systems and Technologies10.1145/3472410.3472417(28-33)Online publication date: 18-Jun-2021
https://dl.acm.org/doi/10.1145/3472410.3472417
Bo LChe ZZhong HXiao Y(2021)A ranking based multi-view learning method for positive and unlabeled graph classificationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.3119626(1-1)Online publication date: 2021
https://doi.org/10.1109/TKDE.2021.3119626
Liu HGuo CCui YShen GPing Y(2021)2-SPIFF: a 2-stage packer identification method based on function call graph and file attributesApplied Intelligence10.1007/s10489-021-02347-w51:12(9038-9053)Online publication date: 1-Dec-2021
https://dl.acm.org/doi/10.1007/s10489-021-02347-w
Zagi LAziz B(2020)Searching for Malware Dataset: a Systematic Literature Review2020 International Conference on Information Technology Systems and Innovation (ICITSI)10.1109/ICITSI50517.2020.9264929(375-380)Online publication date: 19-Oct-2020
https://doi.org/10.1109/ICITSI50517.2020.9264929

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten