TreeOptimizer: A classifier-based task scheduling framework

Distributed Computing (DC) involves a collection of tasks (or modules) executed in parallel on different compute nodes connected through a network. Cloud Service providers (CSP) such as Azure[1], Amazon[2], and Google[3] are providing DC platforms as PaaS (Platform As A Service) offerings. These cloud platforms reduce implementation costs but have a significant drawback as these services can be configured to spawn only a single type of compute node for executing all the tasks in the DC environment. These drawback lead to inefficiency in execution cost and time as each task will have specific compute node requirements. This paper presents a novel framework called TreeOptimizer(TO) to resolve these shortcomings. TO uses a classifier-based dynamic task scheduling to determine the best available node to perform the task. The framework has been tested in Azure Batch[1] for an Oil Industry use case for extracting data from scanned images. Experimental results indicate that TO significantly reduces the overall execution cost by 68% and processing time by 8%. Although this paper uses Batch Service to explain the proposed framework, it can be applied to other PaaS DC platforms.