Dynamic Framework for Reconfiguring Computing Resources in the Inter-cloud and Its Application to Genome Analysis Workflows

Abstract

This paper proposes a framework that dynamically reconfigures an application environment by adding and removing computing resources during runtime. The main idea is that the conditions for the resources used for reconfiguration can be translated into constraints on specifications, such as the number of cores, memory size, and resource location. Our framework consists of two subsystems: an application scheduler, which determines the constraints on specifications for each application, and a resource allocator, which finds resources that satisfy the constraints established by the application scheduler. This structure enables us to apply various reconfiguration strategies by replacing the application scheduler, and also enables us to investigate new allocation strategies for the resource allocator.

As an example of the proposed framework, we developed a reconfiguration module for Galaxy, a workflow manager used in the bioinformatics field. Galaxy can act as an application scheduler by interacting with the reconfiguration module and Galaxy users can take advantage of our reconfiguration framework while using their own interface. The application scheduler applies an embedded strategy to decide when reconfiguration is invoked, whereas it can apply different reconfiguration algorithms to determine constraints on specifications by replacing algorithm modules for reconfiguration. We also describe a scheme for collecting resource metrics, such as CPU usage and memory usage, for use by the reconfiguration algorithms. Finally we conducted preliminary experiments to show the reconfiguration during runtime is necessary because the prediction of resource requirements may fail even if the algorithm uses previous execution records.