Alternative Approaches to High-Performance Metacomputing

Abstract

Software frameworks for high-performance computing have long attempted to deal with the dichotomy between parallel programming on tightly-coupled platforms and the benefits of aggregating distributed heterogeneous resources. Solutions will inevitably involve compromise, but middleware architectures can alleviate the discord. The Harness project has approached high-performance distributed computing via a design philosophy that leverages dynamic reconfigurability – in terms of resources, capabilities, and communication fabrics. We provide an overview of Harness, and position its goals in the context of recent and current metacomputing systems. We then describe the salient features of the H2O framework, a core subsystem in the Harness project, and discuss its alternative approach to high-performance metacomputing. H2O is based on a “pluggable” software architecture to enable flexible and reconfigurable distributed computing. A key feature is the provisioning of customization capabilities that permit clients to tailor provider resources as appropriate to the given application, without compromising control or security. Through the use of uploadable “pluglets”, users can exploit specialized features of the underlying resource, application libraries, or optimized message passing subsystems on demand. H2O is supplemented by subsystems for event handling and fault-tolerant naming services, thereby providing a comprehensive suite of software subsystems for robust and large scale metacomputing. The current status of the H2O subsystem and the overall Harness framework, recent experiences with its use, and planned enhancements are discussed.