Beneath the bytecode: observing the JVM at work using bytecode instrumentation

Many dynamic program analysis (DPA) tools for profiling, debugging, and monitoring programs executing on managed platforms such as the Java Virtual Machine (JVM) rely on bytecode instrumentation (sometimes combined with agents utilizing the JVM Tool Interface and native code libraries) to observe the base program behavior. While this is both the recommended and preferred technique for implementing DPA tools, it has certain noticeable drawbacks [1].

One, the analysis runs in the same process as the base program, and often shares the Java Class Library (JCL) and other resources with the base program. This creates potential for interference that may result in deadlocks, or state corruption in code that does not expect reentrancy.

Two, certain parts of the JCL are typically off-limits for instrumentation, because they either play a vital role during the JVM bootstrap, or the JVM implementation makes certain assumptions about properties of specific classes, or both. These two issues are typically solved by reducing the scope of the instrumentation, leading to under-approximation of the program's behavior.

And three, bytecode instrumentation only allows observing base program events at the bytecode level. The instrumentation code remains oblivious to optimizations performed by the dynamic compiler, and conversely, the compiler is completely unaware of the presence of the instrumentation code. Because the instrumentation code may significantly inflate the base program code and create additional data dependencies as a result of observing the program's behavior, various optimizations performed by the dynamic compiler (e.g., inlining, partial escape analysis, code motion) will be perturbed by the presence of the instrumentation code. As a result, the dynamic analysis may observe events that would not have happened in the base program had it been left alone, thus over-approximating the actual behavior.

In this talk, we will discuss some of the challenges in making the JVM more observable for instrumentation-based DPA tools, with specific focus on getting accurate profiling information in presence of an optimizing dynamic compiler.

The core of this talk is based on the work that was originally presented at OOPSLA'15 [4]. In the meantime, the work has been integrated into the Graal project. Additional parts are based on joint work with other authors, originally presented at AOSD'12 [3] and GPCE'13 [2].