Flexible and Optimized IDL Compilation for Distributed Applications

Abstract

The author of a distributed system is often faced with a dilemma when writing the system’s communication code. If the code is written by hand (e.g., using Active Messages) or partly by hand (e.g., using mpi) then the speed of the application may be maximized, but the human effort required to implement and maintain the system is greatly increased. On the other hand, if the code is generated using a high-level tool (e.g., a corba idl compiler) then programmer efiort will be reduced, but the performance of the application may be intolerably poor. The tradeoff between system performance and development effort arises because existing communication middleware is ineffcient, imposes excessive presentation layer overhead, and therefore fails to expose much of the underlying network performance to application code. Moreover, there is often a mismatch between the desired communication style of the application (e.g., asynchronous message passing) and the communication style of the code produced by an idl compiler (synchronous remote procedure call). We believe that this need not be the case, but that established optimizing compiler technology can be applied and extended to attack these domain-specific problems.

We have implemented Flick, a flexible and optimizing idl compiler, and are using it to explore techniques for producing high-performance code for distributed and parallel applications. Flick produces optimized code for marshaling and unmarshaling data; experiments show that Flickgenerated stubs can marshal data between 2 and 17 times as fast as stubs produced by other idl compilers. Further, because Flick is implemented as a “kit” of components, it is possible to extend the compiler to produce stylized code for many different application interfaces and underlying transport layers. In this paper we outline a novel approach for producing “decomposed” stubs for a distributed global memory service.

This research was supported in part by the Defense Advanced Research Projects Agency, monitored by the Department of the Army under contract number DABT6394C0058, and Air Force Research Laboratory, Rome Research Site, USAF, under agreement number F306029620269. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation hereon.