Characterization of runtime resource usage from analysis of binary executable programs

Highlights

• A method is proposed to characterize a computer program's runtime resource usage from the analysis of its binary executable file.

• The approach produces a phylogeny (hierarchical clustering) of programs based on syntactic similarities found in the executable code.

• Rationales are introduced to explain that these similarities in the binary file correlate to corresponding similarities in runtime behavior.

• The placement of a program in the binary-code phylogeny bears resemblance to the relative similarity of its runtime behavior with respect to the other programs.

• Experimental results show numerically significant evidences that the soft computing method is capable of characterizing runtime profile from binary-code similarities.

Abstract

This paper introduces a methodology for characterizing the runtime resource demands of a computer program from the analysis of its binary executable file. Categorization of applications according to the kind of resources required during execution – such as CPU and memory usage – is a sough-after piece of knowledge for the aims of computer system design and management. Conventional techniques available for this purpose include white-box static source code analysis and profile matching based on historical execution data. The former tends to be challenging in face of complex software architectures and requires access to the source code; the latter is dependent on the availability of reliable past data and on the selection of features yielding effective correlations with resource usage. The alternative data mining approach proposed in this paper avoids those difficulties by manipulating binary executable files. The method combines techniques from information theory, complex networks and phylogenetics to produce a hierarchical clustering of a set of executable files, which can be used to infer potential similarities in terms of runtime resource usage. The paper introduces the method's rationales and presents results of its application to characterize CPU and IO usages of benchmark applications executed on a standard PC platform. Essays carried out over a set of 80 programs from varying sources yielded numerically significant evidences that the prediction of resource usage similarity obtained by the approach is consistent with experimentally measured runtime profile.

Introduction

Knowledge of the platform resources demanded by a program at execution time is a relevant piece of information along different phases of a computer system life-cycle. From capacity planning at the design time, to self-management at operation time, the kind of runtime resources required by an application is a valuable parameter for devising suitable QoS (Quality of Service) strategies, admission control policies, load balancing algorithms and elastic resource allocation mechanisms. In the context of computer programs, if the running process represents the workload submitted to an execution platform, demands occur in the form of usage of CPU, IO, memory, network and other platform resources required by the process. This may be, for instance, a representative scenario of a cluster of processors providing high-performance computation to incoming tasks, or that of a large-scale datacenter serving infrastructure for hosted cloud applications. In both examples there is often room for performance and efficiency optimization through the suitable allocation and adaptive management of resources, for instance, by selecting the best allocation of incoming tasks to service centers so as to improve either response time or energy consumption, or by resizing resource partitions so as to maximize profit from performance versus cost trade-off, and so on. The different patterns of CPU, IO and memory utilization may serve also as system design criterion, specially in embedded systems, e.g. for tuning compiler optimization modes and selecting strategies for reconfigurable FPGA-based hardware [1].

A description of the resource usage of a computer program comprises its runtime resource profile – hereafter referred to simply as its profile. Prevailing methods for performing profile characterization fall into two categories: identification and inference. The former comprises essentially the class of the black-box techniques based on the analysis of data obtained by monitoring the program during execution and directly measuring the process performance parameters of interest. As an experimental method, while not well suited to deriving theoretical formal proofs, it does enjoy the practical confidence of empirical observation. Interactions with the platform and realistic side effects are all present and may be detected. As drawbacks, runtime measurement requires proper instrumentation either in the software or in the platform (operating system or hardware), as well as reasonable guarantees that the potential impact of the instrumental intervention can be estimated and assessed. Moreover, validation of results evokes suitable statistic treatment, what implies in careful experiment design, carrying out enough replications, and ensuring a controllable environment so as to avoid spurious interferences. Naturally, a test bed with all the needed tools must be available.

Alternatively to empirical identification, inference approaches include methods to predict the runtime resource profile from other available clues. One of those is the program code itself. Code analysis may be seen as subcategory under the runtime inference methods based on the inspection of the software code, either manually or by means of automated tools. With the support of formal methods, static code analysis [2] is a deductive approach counting on powerful frameworks to computationally assess functional and non-functional program properties, as well as to prove theoretical hypotheses on correctness and complexity. Nevertheless, code analysis for profile characterization can turn out admittedly difficult and error prone, specially for large programs. Although it can be applied to object code [[3], [4]], e.g. for Java byte-code, code analysis is mainly developed for source code, and as such, it is dependent on the programming language and does not take into account underlying optimizations performed by the compiler, interpreter or the operating system. In addition, concerning static methods, complex interactions with the platform during execution must be deduced by theory, at the risk of leaving behind potentially relevant cross effects. Finally, source code must naturally be available or effectively disassembled from an object code, what is not often practical in many real-world field applications.

In virtue of those intrinsic difficulties, a second subcategory of profile inference methods includes proposed techniques based on pattern recognition. In practice, this inductive approach translates into deciding if a program, whose runtime profile is unknown, is likely to exhibit a given runtime demand by comparing it to similar programs whose profiles have already been characterized. The inherent difficulty at this point is defining “similar”. If the program's functionality is known in advance, knowledge domain may be exploited. For instance, an image processing program is expected to have a runtime profile similar to other programs of the kind, more than to e.g. word processing software. In a private computational environment, where the repertory of client application is restricted and known beforehand, the problem may become relatively easy to handle. Despite advances in this area, however, in a general-purpose environment, like a public cloud computing system, data patterns are not as likely to ensure dependable predictions. In this case, profile inference techniques may resort to past evidences such as log traces. The issue has long being called forth and attempts have been proposed to combine historical data usually available in execution logs such as program name, requesting user and file size [[5], [6], [7]]. Again, in such private environments, those data tend to exhibit some consistence, lending the comparison a reasonable significance, as explored in works addressing shared supercomputer infrastructure [[8], [9]]. In an open public facility, on the other hand, the problem tends to become more challenging. Differing from all the referred methodologies, this paper introduces the application of an unsupervised machine learning method as an alternative approach for predicting the runtime resource usage of a computer program from the analysis of its corresponding executable binary file. With respect to the said problems, the proposed approach tackles profile inference from a data-mining perspective. Analogously to static code analysis, behavior is predicted from the program code; nevertheless, inference is not performed on the semantic level through deductive reasoning, but rather, inductively in the symbolic level through detection of binary code similarities. The rationale of the proposal lies on the intuitive notion that the runtime behavior of a process results from the execution of its algorithm, as expressed by its source code. Being the executable binary code only another representation of the same algorithm, the transformation is expected to preserve the information at some extent. The paper discusses how symbolic patterns existing in the binary file may map runtime features of the executable process, how those hidden clues can be discovered, and how this symbol-to-feature pattern mapping can be exploited to infer the program's resource profile. The method, referred to as Binary Code Clustering (BCC), is based on DAMICORE [10] toolchain. Although lying on non-trivial theoretical foundations, it is of relatively simple implementation.

The approach was applied to characterize the runtime resource patterns of executable programs running on a standard GNU/Linux-based x86 platform. The article addresses the rationales underlying the method and presents a case study with experimental evidences of its success in distinguishing CPU and IO usage profiles of running programs from the data-mining of their corresponding executable files. The paper also outlines potential application of the method in workload classification for performance optimization, as well as foreseen improvements and future works toward them.