Skip to main content
SpringerLink
Log in

Liana: a decentralized load-dependent scheduler for performance-cost optimization of grid service

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This paper addresses the scheduling problem in decentralized grid systems. Such problem focuses on computing a large set of arbitrary tasks to optimize the system performance while minimizing the average system costs. The mainstream solution flourished in recent literatures is to maximize the total system throughput by modeling such systems in either a network flow or a tree. However, most of them neglect the movements of tasks and load-dependent system costs which, in fact, are crucial to the system performance in real situations. In this paper, a Service-Oriented Overlay Network (SOON) is presented, in which the service nodes encapsulate both computation and communication resources and the links are used to track the movements of tasks instead of describing communication. An analytical Cost-Charge (C2) model, in which both running cost and service charge are dependent on load, is proposed to describe the problem by incorporating degree-dependent task allocation into a closed queuing network model. The Infinitesimal Perturbation Analysis (IPA) is applied to solve C2 theoretically. Following the theoretical analysis, a scalable decentralized scheduler named Liana (the movements of tasks in the proposed system like the growth and spread of evergreen liana, so we use Liana to name the proposed scheduler) is proposed. The major components of Liana are an autonomous scheduling algorithm and a Degree-Driven Protocol (DDP). Furthermore, trace based simulations on the test bed distributed widely across the world are implemented to compare the system performance by Liana with recent approaches. The proposed approach shows promising results that the close-to-optimal service utilization is achieved when taking system cost into account.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Foster I, Kesselman C, Tuecke S (2001) The anatomy of the grid: enabling scalable virtual organizations. Int J High Perform Comput Appl 15(3):200–222

    Article  Google Scholar 

  2. Pordes R et al The open science grid. In: Proceedings of the CHEP

  3. Andrade N et al (2003) OurGrid: An approach to easily assemble grids with equitable resource sharing. In: Job scheduling strategies for parallel processing, 2003, pp 61–86

  4. Anderson DP (2004) BOINC: A system for public-resource computing and storage. In: 5th IEEE/ACM international workshop on grid computing, 2004

  5. Hong B, Prasanna V (2007) Adaptive allocation of independent tasks to maximize throughput. IEEE Trans Parallel Distrib Syst 18(10):1420–1435

    Article  Google Scholar 

  6. Beaumont O et al (2006) Centralized versus distributed schedulers for multiple bag-of-task applications. In: 20th International, parallel and distributed processing symposium, IPDPS 2006, 2006

  7. Singh G, Kesselman C, Deelman E (2007) A provisioning model and its comparison with best-effort for performance-cost optimization in grids. In: 16th International symposium on high performance distributed computing, Monterey, California, USA ACM, 2007

  8. Liu D, Cao YD (2007) Queuing network of scale free topology: on performance evaluation of large scale network. Perform Eval (under review)

  9. Xi-Ren C, Han-Fu C (1997) Perturbation realization, potentials, and sensitivity analysis of Markov processes. IEEE Trans Autom Control 42(10):1382–1393

    Article  MATH  Google Scholar 

  10. Ho YC, Cao X (1983) Perturbation analysis and optimization of queueing networks. J Optim Theory Appl 40(4):559–582

    Article  MATH  MathSciNet  Google Scholar 

  11. Barábási A-L, Albert R (1999) Emergence of scaling in random networks. Science 286:509

    Article  MathSciNet  Google Scholar 

  12. Faloutsos M, Faloutsos P, Faloutsos C (1999) On power-law relationships of the Internet topology. In: Proceedings of the conference on applications, technologies, architectures, and protocols for computer communication, Cambridge, Massachusetts, United States, ACM, 1999

  13. Peterson L et al (2002) A blueprint for introducing disruptive technology into the Internet. In: Proceedings of the first ACM workshop on hot topics in networks (HotNets-I), Princeton, ACM, 2002

  14. Beaumont O et al (2002) Bandwidth-centric allocation of independent tasks on heterogeneous platforms. In: Proceedings international parallel and distributed processing symposium, IPDPS 2002, Abstracts and CD-ROM, 2002

  15. Kreaseck B et al (2003) Autonomous protocols for bandwidth-centric scheduling of independent-task applications. In: Proceedings international parallel and distributed processing symposium, 2003

  16. Junwei C et al (2003) Agent-based grid load balancing using performance-driven task scheduling. In: Proceedings international parallel and distributed processing symposium, 2003

  17. Qingjiang W et al. (2006) De-centralized job scheduling on computational Grids using distributed backfilling. Concurr Comput Pract Exp 18(14):1829–1838

    Article  Google Scholar 

  18. Arora M, Das SK, Biswas R (2002) A de-centralized scheduling and load balancing algorithm for heterogeneous grid environments. In: Proceedings international conference on parallel processing workshops, 2002

  19. Elmroth E, Gardfjall P (2005) Design and evaluation of a decentralized system for grid-wide fairshare scheduling. In: First international conference on e-science and grid computing, 2005

  20. Wang C-M et al. (2007) Optimizing server placement in hierarchical grid environments. J Supercomput 42(3):267–282

    Article  Google Scholar 

  21. Litzkow MJ, Livny M, Mutka MW (1988) Condor-a hunter of idle workstations. In: 8th International conference on distributed computing systems, San Jose, CA, IEEE, 1988

  22. Topcuoglu H, Hariri S, Min-You W (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  23. Cormen TH et al. (2001) Introduction to Algorithms, 2nd edn. The MIT Press, Cambridge, London

    MATH  Google Scholar 

  24. Kay J, Lauder P (1988) A fair share scheduler. Commun ACM 31(1):44–55

    Article  Google Scholar 

  25. Chandy KM, Howard JH, Towsley DF (1977) Product form and local balance in queueing networks. J ACM (JACM) 24(2):250–263

    Article  MATH  MathSciNet  Google Scholar 

  26. Henderson R (1995) Job scheduling under the portable batch system. In: Job scheduling strategies for parallel processing, 1995

  27. Brett B et al (2000) The portable batch scheduler and the maui scheduler on Linux clusters. In: Proceedings of the 4th conference on 4th annual Linux showcase & conference. Atlanta, USENIX Association, 2000

  28. HongSoo K et al (2006) Agent-based autonomous scheduling mechanism using availability in desktop grid systems. In: 15th International conference on computing, 2006. CIC ’06, 2006

  29. Sylvia R et al (2001) A scalable content-addressable network. In: Proceedings of the 2001 conference on applications, technologies, architectures, and protocols for computer communications. ACM, San Diego, California, United States, 2001

  30. Barry GL, Evgenia S (2002) Multiple-queue backfilling scheduling with priorities and reservations for parallel systems. SIGMETRICS Perform Eval Rev 29(4):40–47

    Article  Google Scholar 

  31. Mu’alem AW, Feitelson DG (2001) Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans Parallel Distrib Syst 12(6):529–543

    Article  Google Scholar 

  32. Jackson D, Snell Q, Clement M (2001) Core algorithms of the Maui scheduler. In: 7th International workshop on job scheduling strategies for parallel processing, JSSPP 2001, Revised papers, Cambridge, MA, USA, 2001

  33. Barábási A-L, Albert R, Jeong H (2000) Scale-free characteristics of random networks: the topology of the world-wide web. Physica A: Stat Mech Appl 281(1–4):69–77

    Article  Google Scholar 

  34. Mahadevan P et al. (2006) Systematic topology analysis and generation using degree correlations. ACM SIGCOMM Comput Commun Rev 36(4):135–146

    Article  Google Scholar 

  35. Oliveira RV, Zhang B, Zhang L (2007) Observing the evolution of internet as topology. ACM SIGCOMM Comput Commun Rev 37(4):313–324

    Article  Google Scholar 

  36. Gkantsidis C, Mihail M, Saberi A (2003) Conductance and congestion in power law graphs. ACM SIGMETRICS Perform Eval Rev 31(1):148–159

    Article  Google Scholar 

  37. Andrade N et al (2004) Discouraging free riding in a peer-to-peer CPU-sharing grid. In: Proceedings 13th IEEE international symposium on high performance distributed computing, 2004

  38. Amoretti M, Zanichelli F, Conte G (2005) SP2A: a service-oriented framework for P2P-based Grids. In: Proceedings of the 3rd international workshop on middleware for grid computing. Grenoble, France, ACM, 2005

  39. Cirne W et al (2003) Grid computing for bag-of-tasks applications. In: Proceedings of the IFIP I3E2003, 2003

  40. Glasserman P, Gong WB (1990) Smoothed perturbation analysis for a class of discrete-eventsystems. IEEE Trans Autom Control 35(11):1218–1230

    Article  MATH  MathSciNet  Google Scholar 

  41. Evaggelia P et al. (2003) DBGlobe: a service-oriented P2P system for global computing. SIGMOD Rec 32(3):77–82

    Article  Google Scholar 

  42. OGF-OGSI-WG (2004) Open grid service infrastructure, GFD-I. 031, 2004

  43. Banks T (2006) Web Services Resource Framework (WSRF) v1.2. In: wsrf-primer-1.2-primer-cd-02, May 2006, OASIS

  44. Riska A et al (2002) ADAPTLOAD: effective balancing in clustered web servers under transient load conditions. In: Proceedings 22nd international conference on distributed computing systems, 2002

  45. Liu D, Cao Y (2007) CGA: Chaotic genetic algorithm for fuzzy job scheduling in grid environment. Lecture notes in computer science, vol 4456, p 133

  46. Czajkowski K, Foster I, Kesselman C (2005) Agreement-based resource management. Proc IEEE 93(3):631–643

    Article  Google Scholar 

  47. Park K, Pai VS (2006) CoMon: a mostly-scalable monitoring system for PlanetLab. ACM SIGOPS Oper Syst Rev 40(1):65–74

    Article  Google Scholar 

  48. Parallel workloads archive. http://www.cs.huji.ac.il/labs/parallel/workload/index.html

Download references

Author information

Authors and Affiliations

  1. Beijing Laboratory of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology, Beijing, 100081, People’s Republic of China

    Dan Liu, Yuan-Da Cao & Chun-Qing Li

  2. School of Software, Beijing Institute of Technology, Beijing, 100081, People’s Republic of China

    Yuan-Da Cao

Authors
  1. Dan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan-Da Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chun-Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dan Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, D., Cao, YD. & Li, CQ. Liana: a decentralized load-dependent scheduler for performance-cost optimization of grid service. J Supercomput 49, 127–156 (2009). https://doi.org/10.1007/s11227-008-0249-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-008-0249-2

Keywords

Search

Navigation