Fast and Efficient Malware Detection with Joint Static and Dynamic Features Through Transfer Learning

Abstract

In malware detection, dynamic analysis extracts the runtime behavior of malware samples in a controlled environment and static analysis extracts features using reverse engineering tools. While the former faces the challenges of anti-virtualization and evasive behavior of malware samples, the latter faces the challenges of code obfuscation. To tackle these drawbacks, prior works proposed to develop detection models by aggregating dynamic and static features, thus leveraging the advantages of both approaches. However, simply concatenating dynamic and static features raises an issue of imbalanced contribution due to the heterogeneous dimensions of feature vectors to the performance of malware detection models. Yet, dynamic analysis is a time-consuming task and requires a secure environment, leading to detection delays and high costs for maintaining the analysis infrastructure. In this paper, we first introduce a method of constructing aggregated features via concatenating latent features learned through deep learning with equally-contributed dimensions. We then develop a knowledge distillation technique to transfer knowledge learned from aggregated features by a teacher model to a student model trained only on static features and use the trained student model for the detection of new malware samples. We carry out extensive experiments with a dataset of $86\,709$ samples including both benign and malware samples. The experimental results show that the teacher model trained on aggregated features constructed by our method outperforms the state-of-the-art models with an improvement of up to $2.38\%$ in detection accuracy. The distilled student model not only achieves high performance ($97.81\%$ in terms of accuracy) as that of the teacher model but also significantly reduces the detection time (from $70\,046.6$ ms to 194.9 ms) without requiring dynamic analysis.

Appendices

A Ablation Studies

Table 7. Performance (in percentage) of API-ARG student models transferred with different loss functions

1.1 A.1 Transfer Learning for API-ARG Features

Even though transferring knowledge to a student model trained with dynamic features is not a focus of our work, we evaluate the performance of the distilled student models trained with dynamic API-ARG features in this section for completeness. Since the training time for a student model trained with API-ARG features is much longer than that for the models trained with EMBER features or OPCODE features, we only evaluate one set of the recommended hyper-parameters from [13]: $\alpha =0.1, \tau =5$ for KD-KL, and $\alpha =0.1$ for KD-MSE. As shown in Table 7, the distilled student models for API-ARG features perform better than the student-alone model. The performance trends also follow our conjecture that knowledge distillation helps transfer knowledge to a distilled student model with a dynamic API-ARG feature vector that positively contributes to the teacher model (i.e., Agg2-Lat). We also observe that the KD-KL student model obtains better performance compared to the KD-MSE student model, which is different from the results of the case where the student model is trained with EMBER features. This explains why we explored two loss functions during transfer learning for completeness, even though in [16] the authors stated that the KL-loss function is more suitable for a noisy label case, which is not our case since the labels of the malware samples are stabilized and provided by the security vendor.

Table 8. Accuracy (in percentage) and size (in MB) of models with different types of basic blocks (see Fig. 8) for EMBER, OPCODE, and aggregated feature vectors

Table 9. Detailed network architecture for EMBER feature vector with $2\,381$ dimensions

1.2 A.2 Experiments with Different Neural Network Architectures

Besides ResNet1D used for all the experiments presented above, we adopt recent advanced neural network architectures of ResNet in computer vision such as ResNeXt [33], inverted ResNeXt [33], and recently ConvNeXt [18]. We adjust our ResNet1D basic block and implement its variants, namely ResNeXt1D, Inverted ResNeXt1D, and ConvNeXt1D basic blocks accordingly, presented in Fig. 8. We reuse the high-level neural network architectures as defined for ResNet1D in Table 9, Table 10, and Table 11 for each individual feature vector (i.e., EMBER, OPCODE, and API-ARG).

The average performance over 5 runs of ResNet1D (with kernel size $K=3$ and $K=1$) and other variants of the student-alone models trained with EMBER, OPCODE features, and the teacher model trained with Agg2-Lat feature vector are presented in Table 8. We observe that ResNet1D with kernel size $K=3$ obtains the highest accuracy and small size for all the models trained with EMBER, OPCODE, and Agg2-Lat feature vectors. This explains why we chose ResNet1D with kernel size $K=3$ as our basic block for all the experiments presented above.

B Implementation of Neural Network Architecture for Individual Feature Vectors

Detailed implementation of neural network architectures for the EMBER feature vector, OPCODE feature vector and API-ARG feature vector are shown in Table 9, Table 10, and Table 11, respectively.

Table 10. Detailed network architecture for OPCODE feature vector with $33\,338$ dimensions

Table 11. Detailed network architecture for API-ARG feature vector with $1\,048\,576$ dimensions

C Neural Network Architecture of Aggregated Original Features

In Table 12 and Table 13, we present the architectures of the 1D-CNN models for aggregated feature vectors from 2 original feature vectors (Agg2-Org—EMBER + API-ARG) and 3 original feature vectors (Agg3-Org—EMBER + OPCODE + API-ARG), which are developed and evaluated in our work.

D Speeding up Dynamic Analysis with Distributed Cuckoo Infrastructure

As we discussed earlier, it is a time-consuming task of dynamic analysis to produce analysis reports for malware samples. It depends on the user-defined parameter of the longest time that a sample is analyzed in the Cuckoo sandbox but the default setting is two minutes. Based on our daily experiments, using a single Cuckoo sandbox, we were able to analyze 8000 samples per week. We could use multiple sandboxes and manually submit malware samples to speed up the analysis. However, this is quite laborious as the virtual machines hosting Cuckoo sandboxes frequently crash, thus requiring close monitoring for fixing occurring issues.

Table 12. Detailed network architectures for Agg2-Org feature vector ($2\,381 + 1\,048\,576 = 1\,050\,957$ dimensions)

Table 13. Detailed network architectures for Agg3-Org feature vector ($2\,381+33\,338+1\,048\,576=1\,084\,295$ dimensions)

In this work, we developed a parallel dynamic analysis infrastructure using the preliminary version of distributed Cuckoo [28]. We enriched the infrastructure by adding more automation for fault tolerance, which includes

• A re-submission mechanism to resubmit the samples that have not been successfully analyzed. The number of resubmissions is a user-predefined parameter (e.g., three times).

• A monitoring mechanism to check the status of virtual machines whether they are in normal working conditions or crashed. We implement an active monitoring technique that periodically sends monitoring requests to Oracle VirtualBox [22] to check virtual machine status (Cuckoo uses Oracle VirtualBox to host virtual machines and sandboxes).

• A virtual machine (VM) instantiation mechanism to instantiate new VMs to replace the crashed ones. This process is automatically invoked when the monitoring system detects a crashed VM. This allows us to maximize the utilization of computing resources (i.e., the available VMs hosted on physical servers).

Fig. 8.

Different block architectures for 1D-CNN models.

With the developed infrastructure and the available resources in our lab, we could run up to 12 Cuckoo sandboxes at the same time. The implemented fault-tolerance mechanisms also relieve us from the burden of manual monitoring and submission. With more than $86\,000$ samples used in our experiments (discussed further in Sect. 6), we could complete the analysis in just 10 days.

E Discussion on Model Updating

As new malware samples are introduced and evolve daily, model updating is needed to keep the model up to date, thus being able to handle such new samples. However, we believe that this is worth for separate work to develop new methods for model updating such as online learning and dealing with data drift problems. A naive solution is to retrain the teacher model and re-transfer to the student model. The old and new student models are tested on a new test set, and we only deploy the new student if it outperforms the old one by a certain threshold. We will keep this for our future work.