API Call Based Malware Detection Approach Using Recurrent Neural Network—LSTM

Abstract

Malware variants keep increasing every year as most malware developers tweak existing easily available malware codes to create their custom versions. Though their behaviours are coherent, because of change in signature, static signature-based malware detection schemes would fail to identify such malware. One promising approach for detection of malware is dynamic analysis by observing the malware behaviour. Malware executions largely depend on Application Programming Interface (API) calls they issue to the operating systems to achieve their malicious tasks. Therefore, behaviour-based detection techniques that eye on such API system calls can deliver promising results as they are inherently semantic-aware. In this paper, we have used Recurrent Neural Network’s (RNN) capability to capture long-term features of time-series and sequential data to study the scope and effectiveness of RNNs to efficiently detect and analyze malware and benign based on their behaviour, i.e. system call sequences specifically. We trained the RNN-Long Short Term Memory (LSTM) model to learn from the most informative of sequences from the API-dataset based on their relative ranking based on Term Frequency-Inverse Document Frequency (TF-IDF) recommended features and were able to achieve accuracy as high as 92% in detecting malware and benign from an unknown test API-call sequence.