Skip to main content
Springer Nature Link
Log in

A Deadline and Budget Constrained Scheduling Algorithm for eScience Applications on Data Grids

  • Conference paper
Distributed and Parallel Computing (ICA3PP 2005)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3719))

Abstract

In this paper, we present an algorithm for scheduling of distributed data intensive Bag-of-Task applications on Data Grids that have costs associated with requesting, transferring and processing datasets. The algorithm takes into account the explosion of choices that result due to a job requiring multiple datasets from multiple data sources. The algorithm builds a resource set for a job that minimizes the cost or time depending on the user’s preferences and deadline and budget constraints. We evaluate the algorithm on a Data Grid testbed and present the results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Foster, I., Kesselman, C.: The Grid: Blueprint for a Future Computing Infrastructure. Morgan Kaufmann Publishers, San Francisco (1999)

    Google Scholar 

  2. Hey, T., Trefethen, A.E.: The UK e-Science Core Programme and the Grid. Journal of Future Generation Computer Systems(FGCS) 18, 1017–1031 (2002)

    Article  MATH  Google Scholar 

  3. Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.: The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets. Journal of Network and Computer Applications 23, 187–200 (2000)

    Article  Google Scholar 

  4. Lebrun, P.: The Large Hadron Collider, A Megascience Project. In: 38th INFN Eloisatron Project Workshop on Superconducting Materials for High Energy Colliders, Erice, Italy (1999)

    Google Scholar 

  5. Mahajan, R., Bellovin, S.M., Floyd, S., Ioannidis, J., Paxson, V., Shenker, S.: Controlling high bandwidth aggregates in the network. Computer Communications Review 3 (2002)

    Google Scholar 

  6. Buyya, R., Giddy, J., Abramson, D.: A Case for Economy Grid Architecture for Service-Oriented Grid Computing. In: 10th IEEE International Heterogeneous Computing Workshop (HCW 2001), In conjunction with IPDPS 2001, San Francisco, California, USA (April 2001)

    Google Scholar 

  7. Buyya, R., Giddy, J., Abramson, D.: An Evaluation of Economy-based Resource Trading and Scheduling on Computational Power Grids for Parameter Sweep Applications. In: The Second Workshop on Active Middleware Services (AMS 2000), Pittsburgh, USA (2000)

    Google Scholar 

  8. Casanova, H., Legrand, A., Zagorodnov, D., Berman, F.: Heuristics for Scheduling Parameter Sweep Applications in Grid environments. In: 9th Heterogeneous Computing Systems Workshop (HCW 2000), Cancun,Mexico. IEEE CS Press, Los Alamitos (2000)

    Google Scholar 

  9. Takefusa, A., Tatebe, O., Matsuoka, S., Morita, Y.: Performance Analysis of Scheduling and Replication Algorithms on Grid Datafarm Architecture for High-Energy Physics Applications. In: Proceedings of the 12th IEEE international Symposium on High Performance Distributed Computing(HPDC-12), Seattle, USA. IEEE CS Press, Los Alamitos (2003)

    Google Scholar 

  10. Ranganathan, K., Foster, I.: Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications. In: Proceedings of the 11th IEEE Symposium on High Performance Distributed Computing (HPDC), Edinburgh, Scotland. IEEE Computer Society, Los Alamitos (2002)

    Google Scholar 

  11. Park, S.M., Kim, J.H.: Chameleon: A Resource Scheduler in a Data Grid Environment. In: Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003 (CCGrid 2003), Tokyo, Japan. IEEE CS Press, Los Alamitos (2003)

    Google Scholar 

  12. Kim, S., Weissman, J.: A GA-based Approach for Scheduling Decomposable Data Grid Applications. In: Proceedings of the 2004 International Conference on Parallel Processing (ICPP 2004), Montreal, Canada. IEEE CS Press, Los Alamitos (2003)

    Google Scholar 

  13. Venugopal, S., Buyya, R., Winton, L.: A Grid Service Broker for Scheduling Distributed Data-Oriented Applications on Global Grids. In: Proceedings of the 2nd Workshop on Middleware in Grid Computing (MGC 2004): 5th ACM International Middleware Conference (Middleware 2004), Toronto, Canada (2004)

    Google Scholar 

  14. Maheswaran, M., Ali, S., Siegel, H.J., Hensgen, D., Freund, R.F.: Dynamic Mapping of a Class of Independent Tasks onto Heterogeneous Computing Systems. Journal of Parallel and Distributed Computing(JPDC) 59, 107–131 (1999)

    Article  Google Scholar 

  15. Beaumont, O., Legrand, A., Robert, Y., Carter, L., Ferrante, J.: Bandwidth-Centric Allocation of Independent Tasks on Heterogeneous Platforms. In: Proceedings of the 2002 International Parallel and Distributed Processing Symposium(IPDPS 2002), Fort Lauderdale, California, USA. IEEE CS Press, Los Alamitos (2002)

    Google Scholar 

  16. Stockinger, H., Stockinger, K., Schikuta, E., Willers, I.: Towards a Cost Model for Distributed and Replicated Data Stores. In: 9th Euromicro Workshop on Parallel and Distributed Processing PDP 2001, Mantova, Italy. IEEE Computer Society Press, Los Alamitos (2001)

    Google Scholar 

  17. Dail, H., Casanova, H., Berman, F.: A Decoupled Scheduling Approach for the GrADS Environment. In: Proceedings of the 2002 IEEE/ACM Conference on Supercomputing (SC 2002), Baltimore, USA. IEEE CS Press, Los Alamitos (2002)

    Google Scholar 

  18. Hoschek, W., Jaen-Martinez, F.J., Samar, A., Stockinger, H., Stockinger, K.: Data management in an international data grid project. In: Buyya, R., Baker, M. (eds.) GRID 2000. LNCS, vol. 1971, pp. 77–90. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  19. Vazhkudai, S., Tuecke, S., Foster, I.: Replica Selection in the Globus Data Grid. In: Proceedings of the First IEEE/ACM International Conference on Cluster Computing and the Grid (CCGRID 2001), Brisbane, Australia (2001)

    Google Scholar 

  20. Baru, C., Moore, R., Rajasekar, A., Wan, M.: The SDSC Storage Resource Broker. In: Procs. of CASCON 1998, Toronto, Canada (1998)

    Google Scholar 

  21. Hui, T., Tham, C.: Reinforcement learning-based dynamic bandwidth provisioning for quality of service in differentiated services networks. In: Proceedings of IEEE International Conference on Networks (ICON 2003), Sydney, Australia (2003)

    Google Scholar 

  22. Wolski, R., Spring, N., Hayes, J.: The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing. Journal of Future Generation Computing Systems 15, 757–768 (1999)

    Article  Google Scholar 

  23. Vazhkudai, S., Schopf, J.: Using Regression Techniques to Predict Large Data Transfers. International Journal of High Performance Computing Applications 17, 249–268 (2003)

    Article  Google Scholar 

  24. Faerman, M., Su, A., Wolski, R., Berman, F.: Adaptive Performance Prediction for Distributed Data-Intensive Applications. In: Proceedings of the 1999 IEEE/ACM Conference on Supercomputing (SC 1999), Portland, Oregon, USA. IEEE CS Press, Los Alamitos (1999)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Grid Computing and Distributed Systems (GRIDS) Laboratory, Department of Computer Science and Software Engineering, The University of Melbourne, Australia

    Srikumar Venugopal & Rajkumar Buyya

Authors
  1. Srikumar Venugopal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajkumar Buyya
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Information Technology, Geelong, Deakin University, 3217, Vic, Australia

    Michael Hobbs

  2. School of Engineering and Information Technology, Deakin University, Pigdons Road, Geelong

    Andrzej M. Goscinski

  3. Deakin University, Melbourne, Australia

    Wanlei Zhou

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Venugopal, S., Buyya, R. (2005). A Deadline and Budget Constrained Scheduling Algorithm for eScience Applications on Data Grids. In: Hobbs, M., Goscinski, A.M., Zhou, W. (eds) Distributed and Parallel Computing. ICA3PP 2005. Lecture Notes in Computer Science, vol 3719. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564621_7

Download citation

  • DOI: https://doi.org/10.1007/11564621_7

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-29235-7

  • Online ISBN: 978-3-540-32071-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation