Self-describing files + smart modules= parallel program visualization

Abstract

Many tools have been built for gathering and visualizing parallel program performance data, but all too often these tools are (1) tied closely to a specific programming system or computing model and (2) monolithic and hard to customize. We attack problem (1) by using SDF, a self-describing file format that allows a flexible representation of performance information. An SDF file can be processed as a sequence of named records containing named fields without sacrificing the efficiency normally associated with custom-designed binary data representations. Descriptive annotations can be included to further document the contents of an SDF file.

To attack problem (2), we are implementing a system in which reusable performance analysis and visualization modules can be combined in a wide range of configurations. These modules use SDF for their inputs and outputs and are “smart” in that they automatically use the annotations and mnemonic names present in their input data sets to pick appropriate default behaviors, which can be modified interactively if desired. Key to this design is the classification of performance information into general categories such as “event history” and “histogram,” along with the definition of suitable conventions for annotating SDF files containing information of each type.