Dynamic Whitelisting Using Locality Sensitive Hashing

Abstract

Computer systems may employ some form of whitelisting for execution control, verification, minimizing false positives from other detection methods or other purposes. A legitimate file in a whitelist may be represented by its cryptographic hash, such as a hash generated using an SHA1 or MD5 hash function. Due to the fact that any small change to a file in a cryptographic hash results in a completely different hash, a file with a cryptographic hash in a whitelist may no longer be identifiable in the whitelist if the file is modified even by a small amount. This prevents a target file from being identified as legitimate even if the target file is simply a new version of a whitelisted legitimate file.

Locality Sensitive Hashing is a state of the art method in big data and machine learning for the scalable application of approximate nearest neighbor search in high dimensional spaces [9]. The identification of executable files which are very similar to known legitimate executable files fits very well within this paradigm.

In this paper, we show the effectiveness of applying TLSH [1, 2]; Trend Micro’s implementation of locality sensitive hashing, to identify files similar to legitimate executable files. We start with a brief explanation of locality sensitive hashing and TLSH. We then proceed with the concept of whitelisting, and describe typical modifications made to legitimate executable files such as security updates, patches, functionality enhancements, and corrupted files. We will also describe the scalability problems posed by all the legitimate executable files available on the Windows OS. We will also show results of similarity testing against malicious files (malwares). Data will be provided on the efficacy and scalability of this approach. We will conclude with a discussion of how this new methodology may be employed in a variety of computer security applications to improve the functionality and operation of a computer system. Examples may include whitelisting, overriding malware detection performed by a machine learning system, identifying corrupted legitimate files, and identifying new versions of legitimate files.