IEICE Transactions on Communications
Online ISSN : 1745-1345
Print ISSN : 0916-8516
Regular Section
MapReduce Job Scheduling Based on Remaining Job Sizes
Tatsuma MATSUKITetsuya TAKINE
Author information
JOURNAL RESTRICTED ACCESS

2015 Volume E98.B Issue 1 Pages 180-189

Details
Abstract

The MapReduce job scheduler implemented in Hadoop is a mechanism to decide which job is allowed to use idle resources in Hadoop. In terms of the mean job response time, the performance of the job scheduler strongly depends on the job arrival pattern, which includes job size (i.e., the amount of required resources) and their arrival order. Because existing schedulers do not utilize information about job sizes, however, those schedulers suffer severe performance degradation with some arrival patterns. In this paper, we propose a scheduler that estimates and utilizes remaining job sizes, in order to achieve good performance regardless of job arrival patterns. Through simulation experiments, we confirm that for various arrival patterns, the proposed scheduler achieves better performance than the existing schedulers.

Content from these authors
© 2015 The Institute of Electronics, Information and Communication Engineers
Previous article Next article
feedback
Top