Efficient feature extraction methodologies for unknown MP4-Malware detection using Machine learning algorithms

Abstract

We are living in an era in which daily interaction between individuals and businesses involves sending, uploading, and sharing videos as a means of communication and advertising. However, many users are unaware of the risks associated with opening a malicious video file, it is thus no surprise that cyber-criminals have taken advantage of this situation and adopted this attack vector in recent years. MP4 is one of the most commonly used video formats, and its properties make it well-suited for software vulnerability exploitation across multiple platforms, which can ultimately lead to a cyberattack. Due to their deterministic, signature-based technique, antivirus software solutions are limited in their ability to detect unknown malware, let alone zero-day attacks. Machine learning (ML) algorithms have been effective in detecting known and unknown malware across various file formats, domains, and platforms. ML algorithms’ performance relies heavily on the feature extraction methodology. However, to the best of our knowledge, there is no designated and specialized feature extraction methodology for MP4 files which generates a set of features for the task of unknown MP4 file malware detection. In this paper, we present three innovative and efficient feature extraction methodologies for unknown MP4 file malware detection. Two of them are file structure-based and one is knowledge-based. The methodologies are evaluated in a series of five experiments using six ML algorithms and 177 different datasets which represent different configurations of feature extraction, representation, and selection. The datasets are based on a representative collection of 6,229 files − 5,066 benign (∼81.3 %) files and 1,163 malicious files (∼18.7 %). The first three experiments demonstrate the methodologies’ discrimination and generalization capabilities across multiple configurations, in terms of known and unknown MP4 file malware detection. The fourth experiment shows that applying principal component analysis (PCA) on the features suggested by the methodologies can improve time and space complexity and feature resilience while maintaining strong detection and generalization capabilities. In the fifth experiment, the methodologies’ best performing configuration is compared to state-of-the-art, generic feature extraction methodologies, such as n-grams, MinHash, and representation and transfer learning (using a CNN), in the task of unknown MP4 file malware detection. The results show that our best performing configuration outperforms all other state-of-the-art feature extraction methodologies with an AUC, TPR, and FPR of 0.9951, 0.976, and 0.0 respectively.

Introduction

In mid-November 2019, Facebook made a discloser¹ regarding a severe vulnerability in the WhatsApp messenger app - CVE-2019-11931.² The vulnerability allowed attackers to remotely execute arbitrary code or cause a denial-of-service³ (DoS) attack using a specially crafted MP4 file. The exploit stemmed from the way the app parsed elementary metadata streams. Facebook did not reveal many technical details, although examples can be found online⁴ Facebook also didn’t reveal whether this was a zero-click attack,⁵ i.e., the device is infected without the need for any user action in the process. In this case, over 1.5 billion active users were at immediate risk. The above scenario is just one example of how a media file, can be used as an attack vector. In July 2019, Symantec discovered another attack vector called Media File Jacking⁶ which allowed attackers to modify videos and images, and change their content without the users’ knowledge, on both WhatsApp and Telegram; such an attack enables the attacker to bypass end-to-end encryption and potentially transform a user trusted video into a malicious one. The latter example demonstrates how attackers have exploited popular communication and entertainment trends associated with today’s combination of attractive content and high-speed connections, in which users download and share MP4 files without considering the potential malicious outcomes; attackers use social engineering to lure and manipulate victims and cause them to execute and propagate malicious files. For instance, in 2014 Trojan.FakeFlash.A⁷ malware started appearing on Facebook ads promoting naked videos of other Facebook friends. There were no naked videos, but the ad led users to a fake YouTube page aimed at encouraging them to install a fake Adobe Flash Player update to be able to see the videos. This malware infected two million users’ devices directly and convinced many more to at least click on the ad.

Videos are non-executable file formats which users generally consider safer than executable formats. This misconception stems from the fact that non-executable files can only be decoded using dedicated programs, i.e., malicious video files can only exploit vulnerabilities of programs that are devoted to decoding their format. However, once a vulnerability is exploited, non-executable files are as dangerous as executable files, allowing an attacker to perform any malicious actions needed. Moreover, media players are frequently used software. Security Intelligence⁸ reported that over 1,200 vulnerabilities were discovered in the National Vulnerabilities Database⁹ (NVD) from 2000 through 2014. At the time of this writing, we identified 155 published software vulnerabilities directly related to MP4, which were published from 2014 through late 2019. Note that 105 of those vulnerabilities were published since 2017, indicating the increasing risk associated with this attack vector.

MP4 is among the most used media file formats. Therefore, it has nearly unlimited potential of being used to execute a widespread attack. There are no direct available statistics regarding the popularity of MP4 file format, nevertheless, some correlated statistics might. For example, Ecoding.com (one of the world’s largest cloud-based media processing providers), reported¹⁰ that the mostly codec for over-the-top¹¹ (OTT) content is H. 264 which generally refers to MP4 files. Another example is that the recommended¹² format for uploading a video to YouTube is MP4. 207 TB, and 432,000 h of new content is estimated¹³ to be uploaded to YouTube every day. Moreover, MP4 is a commonly used format for uploading videos in most social networks.

Today, common antivirus software uses signature-based¹⁴ malware detection; known for identifying known malware. Unfortunately, new variants of malicious samples appear in the wild every day. Hence, new malware, or even new variants of known malware, could be classified as benign using signature-based methodologies.

Taking into account the above, we would like to highlight three main factors that motivated our study: (1) MP4 is among the most popular media formats; (2) MP4 is a non-executable format, which users often prefer to use, since it cannot be decoded without a dedicated application; and (3) there are no machine learning-based methods or tools that address the challenge of unknow MP4 detection; and as far as we could find there are no publicly available studies (comprehensive or partial) that have been conducted in this domain.

Following the abovementioned, it is surprising that until now, no comprehensive (or partial) studies have been conducted on the paper’s domain. In this study, we present three novel methodologies for the efficient extraction of features from MP4 files, which can be effectively used to train machine learning (ML) algorithms for the purpose of unknown MP4 file malware detection. This paper’s contributions are:

• Proposing three novel feature extraction methodologies for known and unknown MP4 file malware detection.

• Implementing our proposed feature extraction methodologies using different ML algorithms for detection of known and unknown MP4 file malware.

• Determining the best configurations of feature extraction, feature representation, feature selection, top N selected features, and ML algorithms for unknown MP4 file malware detection.

• Improving our proposed methodologies with PCA, in terms of time and space complexity. PCA over the features creates encapsulation and resilience, yet still maintains a high level of generalization. capabilities

• Comparing the best performing configurations with state-of-the-art feature extraction methodologies on the task of unknown MP4 file malware detection based on performance and processing efficiency.

• Detection of unknown MP4 file malware across multiple platforms: Windows, Mac, Linux, Android, etc.

• Creation of a representative up-to-date collection of malicious and benign MP4 files for future research.

The rest of the paper is organized as follows. Section 2 presents an overview of the MP4 file structure, common vulnerabilities, and a possible attack scenario. Section 3 surveys related work on ML-based non-executable files unknown malware detection. Section 4 describes the methods and tools we applied to build a framework aimed at improving unknown MP4 file malware detection capabilities. Section V discusses the evaluation metrics used and the experiments conducted to assess the proposed methodologies, and presents the results obtained. In section 6 we discuss our findings in relation to the methodology’s performance. We conclude in sections VII-IX by discussing the methodologies’ limitations and future work.