Cited By
View all- Fangfei Chen Kodialam MLakshman T(2012)Joint scheduling of processing and Shuffle phases in MapReduce systems2012 Proceedings IEEE INFOCOM10.1109/INFCOM.2012.6195473(1143-1151)Online publication date: Mar-2012
On a local protocol for concurrent file transfers
Authors: 
MohammadTaghi Hajiaghayi, 
Rohit Khandekar, Guy Kortsarz, Vahid LiaghatAuthors Info & Claims
MohammadTaghi Hajiaghayi, Rohit Khandekar, Guy Kortsarz, Vahid LiaghatAuthors Info & ClaimsPages 269 - 278
Abstract
We study a very natural local protocol for a file transfer problem. Consider a scenario where several files, which may have varied sizes and get created over a period of time, are to be transferred between pairs of hosts in a distributed environment. Our protocol assumes that while executing the file transfers, an individual host does not use any global knowledge; and simply subdivides its I/O resources equally among all the active file transfers at that host at any point in time. This protocol is motivated by its simplicity of use and its applications to scheduling map-reduce workloads.
Here we study the problem of deciding the start times of individual file transfers to optimize QoS metrics like average completion time or MakeSpan. To begin with, we show that these problems are NP-hard. We next argue that the ability of scheduling multiple concurrent file transfers at a host makes our protocol stronger than previously studied protocols that schedule a sequence of matchings, in which no two active file transfers share a host at any time. We then generalize the approach of Queyranne and Sviridenko (J. Scheduling, 2002) and Gandhi et al. (ACM T. Algorithms, 2008) that relates the MakeSpan and completion time objectives and present constant factor approximation algorithms.
References
[1]
J.A. Bondy and U.S.R. Murty. Graph Theory.Graduate Texts in Mathematics, 244. Springer, New York, 2008.
[2]
M.M. Halldórsson, G. Kortsarz and M. Sviridenko. Min Sum Edge Coloring in Multigraphs Via Configuration LP.In Proc. 13th Conf. Integer Prog. Combin. Optimiz. (IPCO), 2008.
[3]
R. Gandhi, M. M. Halldórsson, G. Kortsarz and H. Shachnai. Improved Bounds for Scheduling Conflicting Jobs with Minsum Criteria.ACM Transactions on Algorithms. Vol. 4, No. 1, 2008.
[4]
M.M. Halldórsson and G. Kortsarz. Tools for multicoloring with applications to planar graphs and partial k-trees.Journal of Algorithms 42, 2, 334--366, 2002.
[5]
R. Gandhi and J. Mestre. Combinatorial Algorithms for Data Migration to Minimize Average Completion Time.Algorithmica 54, 1,pp 54--71, 2009.
[6]
Y. Kim. Data Migration to Minimize the Average Completion Time.Journal of Algorithms,55:42--57, 2005.
[7]
M.K. Goldberg,phEdge-coloring of multigraphs: recoloring technique.J. Graph Theory, 8:121--137, 1984
[8]
D.S. Hochbaum, T. Nishizeki, and D.B. Shmoys. A better than "Best Possible" algorithm to edge color multigraphs.Journal of Algorithm 7:79--104, 1986.
[9]
E. G. Coffman, M. R. Garey, D. S. Johnson, and A. S. Lapaugh. Scheduling file transfers.SIAM Journal on Computing, 14(3):744--780, 1985.
[10]
A. Bar-Noy, M. Bellare, M. M. Halldórsson, H. Shachnai, and T. Tamir. On chromatic sums and distributed resource allocation. Information and Computation, Vol. 140, pp. 183--202, 1998.
[11]
R. Gandhi, M. M. Halldórsson, G. Kortsarz, and H. Shachnai. Improved Results for Data Migration and Openshop Scheduling.ACM Transactions on Algorithms, 2(1):116--129, 2006.
[12]
M. M. Halldorsson, G. Kortsarz, and H. Shachnai. Sum Coloring Interval Graphs and k-Claw Free Graphs with Applications for Scheduling Dependent Jobs.Algorithmica, 37:187--209, 2003.
[13]
P. Sanders and D. Steurer. An Asymptotic Approximation Scheme for Multigraph Edge Coloring.Proc. of the 16th ACM-SIAM Symposium on Discrete Algorithms, 2005.
[14]
R. Graham. Bounds for certain multiprocessing anomalies.Bell System Technical Journal, 45:15631581, 1966.
[15]
S. Chakrabarti, C. A. Phillips, A. S. Schulz, D. B. Shmoys, C. Stein, and J. Wein. Improved Scheduling Problems For Minsum Criteria.Proc. of the 23rd International Colloquium on Automata, Languages, and Programming, LNCS 1099, 646--657, 1996.
[16]
H. Hoogeveen, P. Schuurman, and G. Woeginger. Non-approximability Results For Scheduling Problems with Minsum Criteria.Proc. of the 6th International Conference on Integer Programming and Combinatorial Optimization, LNCS 1412, 353--366, 1998.
[17]
M. Queyranne and M. Sviridenko. A $(2 + ε)-Approximation Algorithm for Generalized Preemptive Open Shop Problem with Minsum Objective.Journal of Algorithms, 45:202--212, 2002.
[18]
M. Queyranne and M. Sviridenko. Approximation Algorithms for Shop Scheduling Problems with Minsum Objective.Journal of Scheduling, 5:287--305, 2002.
[19]
E. Anderson, J. Hall, J. Hartline, M. Hobbes, A. Karlin, J. Saia, R. Swaminathan, and J. Wilkes. An Experimental Study of Data Migration Algorithms.Workshop on Algorithm Engineering, pages 145--158, 2001.
[20]
J. Hall, J. Hartline, A. Karlin, J. Saia, and J. Wilkes. On Algorithms for Effcient Data Migration.Proc. of the 12th ACM-SIAM Symposium on Discrete Algorithms, pages 620--629, 2001.
[21]
S. Khuller, Y. Kim, and Y. C. Wan. Algorithms for Data Migration with Cloning.In Proc. of the 22nd ACM Symposium on Principles of Database Systems, pages 27--36, 2003.
[22]
Dániel Marx,phMinimum sum multicoloring on the edges of trees.Theor. Comput. Sci., volume 361, number 2--3, pages 133--149, 2006.
[23]
Dániel Marx,phMinimum sum multicoloring on the edges of planar graphs.WAOA, pages 9--22, 2004
[24]
M. Garey and D. Johnson. Computer and intractability. A guide to the theory of NP-completenessFreeman, 1979.
[25]
J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters.In Proc. of the 6th Symposium on Operating System Design and Implementation, pages 137--150, 2004.
Index Terms
- On a local protocol for concurrent file transfers
Recommendations
On a Local Protocol for Concurrent File Transfers
We study a very natural local protocol for a file transfer problem. Consider a scenario where several files, which may have varied sizes and get created over a period of time, are to be transferred between pairs of hosts in a distributed environment. ...
Single machine batch scheduling problem with family setup times and release dates to minimize makespan
In this paper we consider the single machine batch scheduling problem with family setup times and release dates to minimize makespan. We show that this problem is strongly NP-hard, and give an $$O(n (\frac{n}{m}+1)^m)$$ time dynamic programming algorithm and an $$O(mk^{k}P^{2k-1})$$ time dynamic ...
Minimizing makespan on a single batching machine with release times and non-identical job sizes
We consider the problem of scheduling jobs with release times and non-identical job sizes on a single batching machine; our objective is to minimize makespan. We present an approximation algorithm with worst-case ratio 2+@e, where @e>0 can be made ...
Comments
Information & Contributors
Information
Published In
June 2011
404 pages
Copyright © 2011 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
In-Cooperation
- EATCS: European Association for Theoretical Computer Science
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 June 2011
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
SPAA '11: 23rd ACM Symposium on Parallelism in Algorithms and Architectures
June 4 - 6, 2011
California, San Jose, USA
Acceptance Rates
Overall Acceptance Rate 447 of 1,461 submissions, 31%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 143Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
Cited By
View all- Fangfei Chen Kodialam MLakshman T(2012)Joint scheduling of processing and Shuffle phases in MapReduce systems2012 Proceedings IEEE INFOCOM10.1109/INFCOM.2012.6195473(1143-1151)Online publication date: Mar-2012
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in