Diversity/Parallelism Trade-Off in Distributed Systems With Redundancy | IEEE Journals & Magazine | IEEE Xplore

Diversity/Parallelism Trade-Off in Distributed Systems With Redundancy


Abstract:

Distributed computing enables parallel execution of smaller tasks that make up a large computing job. Its purpose is to reduce the job completion time. However, random fl...Show More

Abstract:

Distributed computing enables parallel execution of smaller tasks that make up a large computing job. Its purpose is to reduce the job completion time. However, random fluctuations in task service times lead to straggling tasks with long execution times. Redundancy provides diversity that allows job completion when only a subset of redundant tasks is executed, thus removing the dependency on the straggling tasks. Under constrained resources (here, a fixed number of parallel servers), increasing redundancy reduces the available resources for parallelism. In this paper, we characterize the diversity vs. parallelism trade-off and identify the optimal strategy among replication, coding, and splitting, which minimizes the expected job completion time. We consider three common service time distributions and establish three models that describe the scaling of these distributions with the task size. We find that different distributions with different scaling models operate optimally at different redundancy levels, thus requiring very different code rates.
Published in: IEEE Transactions on Information Theory ( Volume: 68, Issue: 2, February 2022)
Page(s): 1279 - 1295
Date of Publication: 15 November 2021

ISSN Information:

Funding Agency:


References

References is not available for this document.