Efficient Adaptive Resource Management for Spot Workers in Cloud Computing Environment

Abstract

Due to flexible scheduling requirements for various applications, a cloud platform usually has some temporarily unleased workers. To make cost effective, such considerable amount of idle workers can be collected to perform malleable tasks. These workers, called spot workers, often cannot work unstably since they may be interrupted by an external scheduler. This paper proposes a resource management approach that employs replication to increase resource availability. In fact, lifting the replication factor can increase the reliability that a worker can finish a job. But this oppositely decreases resource-usage efficiency due to computational redundancy. We also develop an algorithm that not only controls the replication factor to adapt the changing workload, but also maintains the system performance.