Abstract
Computing intensive scientific workflows structured as a directed acyclic graph (DAG) are widely applied to various distributed science and engineering applications to enable efficient knowledge discovery by automated data processing. Effective mapping and scheduling the workflow modules to the underlying distributed computing environment with heterogeneous resources for optimal network performance has remained as a challenge and attracted research efforts with many simulations and real experiments carried out in the grid and cloud infrastructures. Due to the computing intractability of this type of optimization problem, heuristic algorithms are commonly proposed to achieve the minimum end-to-end delay (EED) or other objectives such as maximum reliability and stability. In this paper, a Hybrid mapping algorithm combining Recursive Critical Path search and layer-based Priority techniques (HRCPP) is designed and developed to achieve the minimum EED. Four representative mapping and scheduling algorithms for minimum EED are compared with HRCPP. Our simulation results illustrate that HRCPP consistently achieves the smallest EED with a low algorithm running time observed from many different scales of simulated test cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kwok, Y., Ahmad, I.: Dynamic critical-path scheduling: An effective technique for allocating task graph to multiprocessors. IEEE Trans. on Parallel and Distributed Systems 7(5) (May 1996)
Wu, Q., Gu, Y.: Supporting distributed application workflows in heterogeneous computing environments. In: Proc. of the 14th IEEE Int. Conf. on Parallel and Distributed Systems, Melbourne, Australia, pp. 3–10 (December 2008)
Sekhar, A., Manoj, B., Murthy, C.: A state-space search approach for optimizing reliability and cost of execution in distributed sensor networks. In: Proc. of Int. Workshop on Distributed Computing, pp. 63–74 (2005)
Agarwalla, B., Ahmed, N., Hilley, D., Ramachandran, U.: Streamline: a scheduling heuristic for streaming application on the grid. In: The 13th Multimedia Computing and Networking Conf., San Jose, CA (2006)
Wu, M.Y., Gajski, D.D.: Hypertool: A programming aid for message-passing systems. IEEE Trans. on Parallel and Distributed Systems 1(3), 330–343 (1990)
Wang, L., Siegel, H.J., Roychowdhury, V.P., Maciejewski, A.A.: Task matching and scheduling in heterogeneous computing environment using a genetic-algorithm-based approach. Journal Parallel and Distributed Computing 4, 175–187 (1997)
Carter, B.R., Watson, D.W., Freund, R.F., Keith, E., Mirabile, F., Siegel, H.J.: Generational scheduling for dynamic task management in heterogeneous computing systems. Information Science 106(3-4), 219–236 (1998)
Ma, T., Buyya, R.: Critical-path and priority based algorithms for scheduling workflows with parameter sweep tasks on global grids. In: Proc. of the 17th Int. Symp. on Computer Architecture on High Performance Computing, pp. 251–258 (2005)
Kwok, Y.K., Ahmad, I.: Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Computing Surveys 31(4), 406–471 (1999)
Afrati, F.N., Papadimitriou, C.H., Papageorgiou, G.: Scheduling dags to minimize time and communication. In: Proc. of the 3rd Aegean Workshop on Computing: VLSI Algorithms and Architectures, pp. 134–138. Springer, Heidelberg (1988)
Topcuoglu, H., Hariri, S., Wu, M.: Performance effective and lowcomplexity task scheduling for heterogeneous computing. IEEE Trans. on Parallel and Distributed Systems 13(3) (2002)
Cordella, L., Foggia, P., Sansone, C., Vento, M.: An improved algorithm for matching large graphs. In: Proc. of the 3rd IAPR-TC-15 Int. Workshop on Graph-based Representations, Italy (2001)
Rahman, M., Venugopal, S., Buyya, R.: Ant colony system: A dynamic critical path algorithm for scheduling scientific workflow applications on global grids. In: Proc. of the 3rd IEEE Int. Conf. on e-Science and Grid Computing, pp. 35–42 (2007)
Boeres, C., Filho, J., Rebello, V.: A cluster-based strategy for scheduling task on heterogeneous processors. In: in Proc. of 16th Symp. on Computer Architecture and High Performance Computing, pp. 214–221 (2004)
Rahman, M., Venugopal, S., Buyya, R.: A dynamic critical path algorithm for scheduling scientific workflow applications on global grids. In: In Proc. of the 3rd IEEE Int. Conf. on e-Science and Grid Computing, pp. 35–42 (2007)
Wu, Q., Gu, Y., Lin, Y., Rao, N.S.V.: Latency Modeling and Minimization for Large-scale Scientific Workflows in Distributed Network Environments. In: Proc. of the 44th Annual Simulation Symposium (ANSS 2011), Boston, MA, USA, April 4-7 (2011)
Zhu, M., Wu, Q., Rao, N.S.V., Iyengar, S.S.: Adaptive visualization pipeline decomposition and mapping onto computer networks. In: Proc. of the IEEE Internatioal Conference on Image and Graphics, Hong Kong, China, December 18-20, pp. 402–405 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhu, M., Cao, F., Mi, J. (2011). A Hybrid Mapping and Scheduling Algorithm for Distributed Workflow Applications in a Heterogeneous Computing Environment. In: Brazier, F.M.T., Nieuwenhuis, K., Pavlin, G., Warnier, M., Badica, C. (eds) Intelligent Distributed Computing V. Studies in Computational Intelligence, vol 382. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24013-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-24013-3_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24012-6
Online ISBN: 978-3-642-24013-3
eBook Packages: EngineeringEngineering (R0)