Loading web-font TeX/Main/Regular
Joint Scheduling and Source Selection for Background Traffic in Erasure-Coded Storage | IEEE Journals & Magazine | IEEE Xplore

Joint Scheduling and Source Selection for Background Traffic in Erasure-Coded Storage


Abstract:

Erasure-coded storage systems have gained considerable adoption recently since they can provide the same level of reliability with significantly lower storage overhead co...Show More

Abstract:

Erasure-coded storage systems have gained considerable adoption recently since they can provide the same level of reliability with significantly lower storage overhead compared to replicated systems. However, background traffic of such systems – e.g., repair, rebalance, backup and recovery traffic – often has large volume and consumes significant network resources. Independently scheduling such tasks and selecting their sources can easily create interference among data flows, causing severe deadline violation. We show that the well-known heuristic scheduling algorithms fail to consider important constraints, thus resulting in unsatisfactory performance. In this paper, we claim that an optimal scheduling algorithm, which aims to maximize the number of background tasks completed before deadlines, must simultaneously consider task deadline, network topology, chunk placement, and time-varying resource availability. We first show that the corresponding optimization problem is NP-hard. Then we propose a novel algorithm, called Linear Programming for Selected Tasks (LPST) to maximize the number of successful tasks and improve overall utilization of the datacenter network. It jointly schedules tasks and selects their sources based on a notion of Remaining Time Flexibility, which measures the slackness of the starting time of a task. We evaluated the efficacy of our algorithm using extensive simulations and validate the results with experiments in a real cloud environment. Our results show that, under certain scenarios, LPST can perform 7x\sim 10x better than the heuristics which blindly treat the infrastructure as a collection of homogeneous resources, and 21.7 \sim65.9 percent better than the algorithms that only take the network topology into account.
Published in: IEEE Transactions on Parallel and Distributed Systems ( Volume: 29, Issue: 12, 01 December 2018)
Page(s): 2826 - 2837
Date of Publication: 11 June 2018

ISSN Information:


Contact IEEE to Subscribe

References

References is not available for this document.