A least flow-time first load sharing approach for distributed server farm

doi:10.1016/j.jpdc.2005.02.007

Journal of Parallel and Distributed Computing

Volume 65, Issue 7, July 2005, Pages 832-842

https://doi.org/10.1016/j.jpdc.2005.02.007 Get rights and content

Abstract

The most critical property exhibited by a heavy-tailed workload distribution (found in many WWW workloads) is that a very small fraction of tasks make up a large fraction of the workload, making the load very difficult to distribute in a distributed system. Load balancing and load sharing are the two predominant load distribution strategies used in such systems. Load sharing generally has better response time than load balancing because the latter can exhibit excessive overheads in selecting servers and partitioning tasks. We therefore further explored the least-loaded-first (LLF) load sharing approach and found two important limitations: (a) LLF does not consider the order of processing, and (b) when it assigns a task, LLF does not consider the processing capacity of servers. The high task size variation that exists in heavy-tailed workloads often causes smaller tasks to be severely delayed by large tasks.

This paper proposes a size-based approach, called the least flow-time first (LFF-SIZE), which reduces the delay caused by size variation while maintaining a balanced load in the system. LFF-SIZE takes the relative processing time of a task into account and dynamically assigns a task to the fittest server with a lighter load and higher processing capacity. LFF-SIZE also uses a multi-section queue to separate larger tasks from smaller ones. This arrangement effectively reduces the delay of smaller tasks by larger ones as small tasks are given a higher priority to be processed. The performance results performed on the LFF-SIZE implementation shows a substantial improvement over existing load sharing and static size-based approaches under realistic heavy-tailed workloads.

Section snippets

References (17)

B. Awerbuch, Y. Azar, S. Leonardi, O. Regev, Minimizing the flow time without migration, Proceedings of ACM Symposium...
J. Broberg, Z. Tari, P. Zeephongsekul, Task assignment based on prioritising traffic flows, RMIT Technical Report...
Cisco Systems, LocalDirector, Documentation available at...
R.W. Conway et al.
Theory of Scheduling
(1967)
M. Crovella, A. Bestavros, Self-similarity in world wide web traffic: evidence and possible causes, Proceedings of ACM...
M. Crovella, M. Harchol-Balter, C. Murta, Task assignment in a distributed system: improving performance by unbalancing...
M. Crovella, M. Taqqu, A. Bestravos, A practical guide to heavy tails, in: R.J. Adler, R.E. Feldman, M.S. Taqqu (Eds.),...
E. Haddad, Load balancing and scheduling in network heterogeneous Computing, in: M.M. Eshaghian (Ed), Heterogeneous...

There are more references available in the full text version of this article.

Cited by (22)

Analysis of the Task Assignment based on Guessing Size policy
2020, Performance Evaluation
We study the Task Assignment based on Guessing Size (TAGS) policy in a parallel and homogeneous server system. The policy parameters are the number of servers $h$ and a set of cutoffs $s_{1} < s_{2}, l d o t s, s_{h - 1}$ . In this policy, all the incoming jobs are routed to the first server and jobs are run up to $s_{1}$ time units. If they complete they leave the system, but jobs that do not complete after $s_{1}$ time units are killed and moved to the end of the queue of the second server, where service starts from scratch. Likewise, jobs that are executed in server $i$ and complete service before $s_{i}$ units of time, leave the system, whereas jobs that do not complete are killed and routed to the next server. We first study the stability of such system and provide a precise utilization threshold for the existence of stable parameters for a given job size distribution. We compute the threshold for several families of distributions and provide bounds for others. We show that TAGS is most stable for bounded Pareto distributions with parameter $α = 1$ . Besides, we provide tight bounds on the performance of the TAGS policy where the cutoffs are chosen to minimize average waiting time in the asymptotic regime where the largest job size tends to infinity for the Bounded Pareto distribution and the system load smaller than one. In this case, we show that the performance ratio between TAGS and a version called SITA which does require knowledge of job size is at most 2. We then consider more broadly the same asymptotic regime and consider a bound on the average waiting time for any distribution. This is compatible with having a conservative policy which will work well for any job size distribution. We show rather tight upper and lower bounds which again match those of SITA up to a factor of 2. These bounds are considerably lower than the corresponding bounds of competing policies such as Random or Least Work Remaining. We also show that for all these policies the bounded Pareto distribution with $α = 1$ is close to being the worst possible among all distributions with the same job size range. We show using the stability results that in the same regime, if we increase the load the gap between SITA and TAGS grows dramatically and eventually, the TAGS system cannot be stabilized. The conclusion from all our analysis is that TAGS is best used as a conservative policy with minimal requirements on small systems with relatively low utilization and where the range of job sizes is large.
Task assignment in multiple server farms using preemptive migration and flow control
2011, Journal of Parallel and Distributed Computing
Citation Excerpt :
Let us first draw our attention to the task assignment models that assume known task sizes. Examples of such models include Size Interval Task Assignment with Equal Load (SITA-E) [21], ADAPTLOAD [40], EQUILOAD [9] and Lest Flow-Time First Load Sharing (LFF-SIZE) [33]. SITA-E improves the performance by processing tasks with similar sizes at the same host in that it has hosts that have specific task size ranges assigned to them.
Existing task assignment policies proposed for assigning tasks in stand-alone server farms are not efficient in multiple server farm environments because they have not been designed to exploit the properties of such environments. With the emergence of high speed networks and operating systems that have features such as preemptive migration, the importance of designing task assignment policies for assigning tasks in multiple server farms has increased. Such policies can result in better overall performance compared to those that optimise performance in stand-alone server farms.
This paper proposes a task assignment policy suitable for assigning tasks in multiple server farms. The proposed policy, called Multi-Cluster Task Assignment based on Preemptive Migration (MCTPM) is based on a multi-tier host architecture that reduces the variance of task sizes in host queues by processing tasks with similar sizes using a set of hosts that have a distinct task size range. MCTPM controls the traffic flow into server farms via a global dispatching device so as to optimise the performance. MCTPM supports preemptive task migration between servers in the same farm and between servers in different farms.
Performance analysis of the proposed policy indicates that significant performance improvements are possible under a wide range of workload scenarios. For example, MCTPM outperforms existing policies such as MC-Random, MC-TAGSPM and MC-MTTPM by factors of $190$ , $5$ and $10.5$ respectively under certain scenarios.
Task assignment policies in distributed server systems: A survey
2011, Journal of Network and Computer Applications
Citation Excerpt :
In recent logs (for 2004), from proxy servers, 67–73% of requests is for static content (IRCacheHome, 2004). Serving static requests quickly is the focus of many companies, e.g. Akamai Technologies, CDNetworks, Certeon technologies, Crescendo Networks, F5 networks, Keynote Systems, Radware technologies, and Riverbed technologies. Therefore, popular Web sites often convert their dynamic content into static content when the Web site is overloaded, which makes static requests especially important (LeFebvre, 2002).
Data intensive computing, in the Web environment, motivates the distributed designs of Web server systems (Web clusters) because of their scalability and cost-effectiveness instead of one Web server with high performance. The task assignment policy, in such systems, focuses on the manner of assigning the tasks that reach these systems (e.g. the case of intensive requests that reach the distributed Web server systems) in order to minimize the response time and thus, improve the performance. These tasks, generally, follow the “heavy-tailed” distribution which has the property that there is a tiny fraction (about 3%) of large tasks that makes half (50%) of the total load. Several policies were proposed in the literature to deal with the nature of this Web traffic. This paper presents a state-of-art of the existing task assignment policies. We classify these policies in two classes: policies which assume that the task size is known a priori, and policies which assume that the task size is not known a priori (like TAGS, TAPTF and TAPTF-WC). The first class of policies regroups policies which consider no knowledge of load information at the servers when assigning the incoming tasks, known as static policies (like Random, Round Robin, etc.) and, policies, known as dynamic policies (like Central Queue Policy, Least Loaded First “LLF”, etc.) which use some load information (e.g. the processing capacity, the queue load, etc.) to process.
To balance or unbalance load in size-interval task allocation
2010, Probability in the Engineering and Informational Sciences
Study on Dynamic Resource Scheduling Method for Domestic Operating System Virtualization Based on Computing Domain
2022, Advances in Transdisciplinary Engineering
Transmission delay prediction based data allocation scheme for concurrent multipath transfer
2015, Ruan Jian Xue Bao/Journal of Software

View all citing articles on Scopus

Zahir Tari is currently an Associate Professor at RMIT University, Melbourne (Australia). He is the director of the “Distributed Systems and Networking” discipline, involving four research groups working in the development of the next generation of distributed systems and applications. Details about the discipline can be found at http://www.cs.rmit.edu.au/eCDS. Dr. Tari has been very active in system performance (e.g. load balancing), security (e.g. access control, information flow control), web services (e.g. communication protocols). He published in IEEE and ACM Transactions, as well as in reputable conferences (e.g. ICDCS, ICDE). Dr. Tari has been the General Chair and Program Committee Chair of more than 10 international conferences. See http://www.cs.rmit.edu.au/~zahirt for more details.

James Broberg is currently a Ph.D. student working under the supervision of Dr. Zahir Tari at RMIT University, Melbourne (Australia). He is a member of the “Distributed Systems and Networking” discipline at RMIT, and has worked and published in the area of task assignment (e.g. scheduling policies) to enable effective load balancing and load sharing in distributed systems. Mr. Broberg has also worked as an intern at IBM Research (T.J. Watson Research Center) working on network flow problems.

Albert Y. Zomaya is currently the CISCO Systems Chair Professor of Internetworking in the School of Information Technologies, The University of Sydney. Prior to that he was a Full Professor in the Electrical and Electronic Engineering Department at the University of Western Australia, where he also led the Parallel Computing Research Laboratory during the period 1990–2002. He served as Associate-, Deputy-, and Acting-Head in the same Department, and held visiting positions at Waterloo University and the University of Missouri-Rolla. He is the author/co-author of 6 books, 200 publications in technical journals and conferences, and the editor of 6 books and 7 conference volumes. He is currently an Associate Editor for 14 journals, the Founding Editor of the Wiley Book Series on Parallel and Distributed Computing, and the Editor-in-Chief of the Parallel and Distributed Computing Handbook (McGraw-Hill, 1996). Professor Zomaya was the Chair the IEEE Technical Committee on Parallel Processing (1999–2003) and currently serves on its executive committee. He has been actively involved in the organization of national and international conferences. He received the 1997 Edgeworth David Medal from the Royal Society of New South Wales for outstanding contributions to Australian Science. In September 2000 he was awarded the IEEE Computer Society's Meritorious Service Award. Professor Zomaya is a Chartered Engineer (CEng), a Fellow of the IEEE, a Fellow of the Institution of Electrical Engineers (UK), and member of the ACM. He also serves on the boards of two startup companies. His research interests are in the areas of high-performance computing, parallel algorithms, networking, and bioinformatics.

Roberto Baldoni holds a Professor position at the University of Rome “La Sapienza”, Italy. He published more than 100 papers (from theory to practice) in the fields of distributed and mobile computing, middleware platforms and information systems. He is the founder of MIDdleware LABoratory (MIDLAB) whose members participate to the following national and european research projects: MIDAS (IST), EU-Publi.com (IST), DaQuincis(MIUR-Cofin), MAIS (MIUR-FIRB) and IS-MANET (MIUR). He regularly serves as an expert for the EU commission in the evaluation of EU projects and was in the organizing and program committee of premiership international conferences and workshops such as ICDCS, DSN, SRDS, DISC, EDDC, PERCOM, EUROPAR, ISORC, DOA and CoopIS.

View full text

Journal of Parallel and Distributed Computing

A least flow-time first load sharing approach for distributed server farm

Abstract

Section snippets

Theory of Scheduling

Analysis of the Task Assignment based on Guessing Size policy

Task assignment in multiple server farms using preemptive migration and flow control

Task assignment policies in distributed server systems: A survey

To balance or unbalance load in size-interval task allocation

Study on Dynamic Resource Scheduling Method for Domestic Operating System Virtualization Based on Computing Domain

Transmission delay prediction based data allocation scheme for concurrent multipath transfer