Skip to main content
SpringerLink
Log in

DDOps: dual-direction operations for load balancing on non-dedicated heterogeneous distributed systems

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Given the increasing prevalence of compute/data intensive applications, the explosive growth in data, and the emergence of cloud computing, there is an urgent need for effective approaches to support such applications in non-dedicated heterogeneous distributed environments. This paper proposes an efficient technique for handling parallel tasks, while dynamically maintaining load balancing. Such tasks include concurrently downloading files from replicated sources; simultaneously using multiple network interfaces for message transfers; and executing parallel computations on independent distributed processors. This technique, DDOps, (Dual Direction Operations) enables efficient utilization of available resources in a parallel/distributed environment without imposing any significant control overhead. The idea is based on the notion of producer pairs that perform tasks in parallel from opposite directions and the consumers that distribute and control the work and receive and combine the results. Most dynamic load balancing approaches require prior knowledge and/or constant monitoring at run time. In DDOps, load balancing does not require prior knowledge or run-time monitoring. Rather, load balancing is automatically inherent as the tasks are handled from the opposite directions, allowing the processing to continue until the producers meet indicating the completion of all tasks at the same time. Thus DDOps is most suitable for heterogeneous environments where resources vary in specifications, locations, and operating conditions. In addition, since DDOps does not require producers to communicate at all, the network effect is minimized.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Adiseshu, H., Parulkar, G., Vargese, G.: A reliable and scalable striping protocol. In: Computer Communication Review, October 1996 vol. 26, pp. 131–141. ACM, New York (1996)

    Google Scholar 

  2. Agarwal, R., Gustavson, F., Zubair, M.: A high performance matrix multiplication algorithm on a distributed-memory parallel computer, using overlapped communication. IBM J. Res. Dev. 38(6), 673–681 (1994)

    Article  Google Scholar 

  3. Al-Azzoni, I., Down, D.G.: Decentralized load balancing for heterogeneous grids. In: Proc. of the International Conference on Future Computing, pp. 545–550 (2009)

    Google Scholar 

  4. Al-Jaroodi, J., Mohamed, N.: DDFTP: dual-direction FTP. In: Proc. of the 11th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing (CCGrid), Newport Beach, CA, USA, May 2011

    Google Scholar 

  5. Al-Jaroodi, J., Mohamed, N., Jiang, H., Swanson, D.: Middleware infrastructure for parallel and distributed programming models on heterogeneous systems. IEEE Trans. Parallel Distrib. Syst. 14(11), 1100–1111 (2003)

    Article  Google Scholar 

  6. Allcock, B., Bester, J., Bresnahan, J., Chervenak, A.L., Kesselman, C., Meder, S., Nefedova, V., Quesnel, D., Tuecke, S., Foster, I.: Secure, efficient data transport and replica management for high-performance data-intensive computing. In: Proc. of the IEEE 18th Symposium on Mass Storage Systems and Technologies, San Diego, CA, USA, April 2001

    Google Scholar 

  7. Alonso, P., Reddy, R., Lastovetsky, A.: Experimental study of six different implementations of parallel matrix multiplication on heterogeneous computational clusters of multicore processors. In: Proc. of the 18th Euromicro Conference on Parallel, Distributed and Network-Based Processing (2010)

    Google Scholar 

  8. Aubanel, E.: Resource-aware load balancing of parallel applications. In: Handbook of Research on Grid Technologies and Utility Computing: Concepts for Managing Large-Scale Applications, May 2009, pp. 12–21. IGI Global, Hershey (2009)

    Chapter  Google Scholar 

  9. Beaumont, O., Boudet, V.: Matrix multiplication on heterogeneous platforms. In: IEEE Transaction on Parallel and Distributed Systems, vol. 12, pp. 1033–1051 (2001)

    Google Scholar 

  10. Beowulf Ethernet channel bonding. http://www.phy.duke.edu/~rgb/brahma/Resources/beowulf/software/bonding.html. Accessed Apr 2013

  11. Bhardwaj, D., Kumar, R.: A parallel file transfer protocol for clusters and grid systems. In: Proc. of the 1st International Conference on e-Science and Grid Computing (e-Science), pp. 248–254 (2005)

    Google Scholar 

  12. Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK: a linear algebra library for message-passing computers. In: Proc. of the 8th SIAM Conference on Parallel Processing for Scientific Computing, (PPSC), Minneapolis, Minnesota, USA, March 1997, pp. 14–17 (1997)

    Google Scholar 

  13. Boman, E.G., Catalyurek, U.V., Chevalier, C., Devine, K.D., Safro, I., Wolf, M.M.: Advances in parallel partitioning, load balancing and matrix ordering for scientific computing. J. Phys. Conf. Ser. 180, 1 (2009)

    Article  Google Scholar 

  14. Channel bonding performance. http://www.scl.ameslab.gov/Projects/MP_Lite/perf_cb_fe.html. Accesssed Apr 2013

  15. Chiasson, J., Tang, Z., Ghanem, J., Abdallah, C.T., Birdwell, J.D., Hayat, M.M., Jérez, H.: The effect of time delays on the stability of load balancing algorithms for parallel computations. IEEE Trans. Control Syst. Technol. 13(6), 932–942 (2005)

    Article  Google Scholar 

  16. Chung, I., Bae, Y.: The design of an efficient load balancing algorithm employing block design. J. Appl. Math. Comput. 14(1–2), 343–351 (2004)

    Article  MathSciNet  Google Scholar 

  17. Configuring Ethernet channel bonding. http://www.linuxtopia.org/online_books/linux_system_administration/redhat_cluster_configuration_and_management/generated-index.html. Accessed Apr 2013

  18. De Giusti, A.E., Naiouf, M.R., De Giusti, L.C., Chichizola, F.: Dynamic load balancing in parallel processing on non-homogeneous clusters. J. Comput. Sci. Technol. 5(4), 272–278 (2005)

    Google Scholar 

  19. Dhakal, S., Hayat, M., Pezoa, J.E., Yang, C., Bader, D.A.: Dynamic load balancing in distributed systems in the presence of delays: a regeneration-theory approach. IEEE Trans. Parallel Distrib. Syst. 18(4), 485–497 (2007)

    Article  Google Scholar 

  20. Eggen, M., Franklin, N., Eggen, R.: Load balancing on a non-dedicated heterogeneous network of workstations. In: Proc. of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA), pp. 856–862 (2002)

    Google Scholar 

  21. Figueira, S., Berman, F.: A slowdown model for applications executing on time-shared clusters of workstations. IEEE Trans. Parallel Distrib. Syst. 12(6), 653–670 (2001)

    Article  Google Scholar 

  22. File transfer protocol, RFC 959. http://www.faqs.org/rfcs/rfc959.html. Accessed Apr 2013

  23. Fox, G., Otto, S., Hey, A.: Matrix algorithms on a hypercube I: matrix multiplication. Parallel Comput. 3, 17–31 (1987)

    Article  Google Scholar 

  24. FTP client software and FTP server software. http://en.wikipedia.org/wiki/Comparison_of_FTP_client_software, http://en.wikipedia.org/wiki/Comparison_of_FTP_server_software. Accessed Apr 2013

  25. Funasaka, J., Kawano, A., Ishida, K.: Implementation issues of parallel downloading methods for a proxy system. In: Proc. of the 4th International Workshop on Assurance in Distributed Systems and Networks (ICDCSW), vol. 1, pp. 58–64 (2005)

    Google Scholar 

  26. GridFTP: universal data transfer for the grid, The Globus Project, documentation. The University of Chicago and The University of Southern CA, USA. http://www.globus.org/alliance/publications/papers.php. Accessed Apr 2013

  27. Hargrove, W., Hoffman, F.: Optimizing master/slave dynamic load balancing in heterogeneous parallel environments. White paper, Oak Ridge National Laboratory, Oak, Ridge, TN (1999)

  28. HPSS user’s guide—high performance storage system, release 7.1, 2009 International Business Machines Corporation, USA, Feb. 2009

  29. Hsu, C.-H., Chu, C.-W., Chou, C.-H.: Bandwidth sensitive co-allocation scheme for parallel downloading in data grid. In: Proc. of the IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA), pp. 34–39 (2009)

    Google Scholar 

  30. Johnsen, E.S., Anshus, O.J., Bjørndalen, J.M., Bongo, L.A.: Survey of optimizing techniques for parallel programs running on computer clusters. Technical report, Department of Computer Science, University of Tromsø, September (2003). http://www.idi.ntnu.no/~elster/notur-cluster03/NOTUR_ET_Cluster_A12b_ifi_uit_survey.pdf. Accessed Apr 2013

  31. Kathiresan, N.: Adaptive distributed load balancing for heterogeneous computing system using global scheduler policy. In: Proc. of the 4th International Working Conference on Performance Modelling and Evaluation of Heterogeneous Networks, UK, September 2006

    Google Scholar 

  32. Katz, R., Gibson, G., Patterson, D.: Disk system architectures for high performance computing. Proc. IEEE 77(12), 1842–1858 (1989)

    Article  Google Scholar 

  33. Kawano, A., Funasaka, J., Ishida, K.: Parallel downloading using variable length blocks for proxy servers. In: Proc. of the 27th International Conference on Distributed Computing Systems Workshops (ICDCSW) (2007)

    Google Scholar 

  34. Khan, S.U., Ahmad, I.: A cooperative game theoretical technique for joint optimization of energy consumption and response time in computational grids. IEEE Trans. Parallel Distrib. Syst. 20(3), 346–360 (2009)

    Article  MathSciNet  Google Scholar 

  35. Kliazovich, D., Bouvry, P., Khan, S.U.: DENS: data center energy-efficient network-aware scheduling. Clust. Comput. 16(1), 65–75 (2013)

    Article  Google Scholar 

  36. Kołodziej, J., Khan, S.U., Wang, L., Byrski, A., Min-Allah, N., Madani, S.A.: Hierarchical genetic-based grid scheduling with energy optimization. Clust. Comput. (1994). doi:10.1007/s10586-012-0226-7

    Google Scholar 

  37. Kumar, V., Grama, A., Gupta, A., Karypis, G.: Introduction to Parallel Computing. Benjamin-Cummings, Redwood City (1994)

    MATH  Google Scholar 

  38. Li, Y., Lan, Z.: A survey of load balancing in grid computing. In: Computational and Information Science. Lecture Notes in Computer Science, vol. 3314, pp. 280–285 (2005)

    Chapter  Google Scholar 

  39. Lim, S.B., Fox, G., Kaplan, A., Pallickara, S., Pierce, M.: GridFTP and parallel TCP support in NaradaBrokering. In: Distributed and Parallel Computing. Lecture Notes in Computer Science, vol. 3719, pp. 93–102 (2005)

    Chapter  Google Scholar 

  40. Mateescu, G.: Parallel sorting on heterogeneous platforms. In: Proc. of the 16th International Symposium on High Performance Computing Systems and Applications, pp. 116–117, June 2002

    Chapter  Google Scholar 

  41. Mohamed, N., Al-Jaroodi, J.: Self-configured multiple-network-interface socket. J. Netw. Comput. Appl. 33(1), 35–42 (2010)

    Article  Google Scholar 

  42. Mohamed, N., Al-Jaroodi, J., Jiang, H., Swanson, D.: A middleware-level parallel transfer technique over multiple network interfaces. In: Proc. of the ClusterWorld Conference & Expo, San Jose, CA, June 2003

    Google Scholar 

  43. Pinar, A., Hendrickson, B.: Improving load balance with flexibly assignable tasks. IEEE Trans. Parallel Distrib. Syst. 16(10), 956–965 (2005)

    Article  Google Scholar 

  44. Pınar, A., Tabak, E.K., Aykanat, C.: One-dimensional partitioning for heterogeneous systems: theory and practice. J. Parallel Distrib. Comput. 68, 1473–1486 (2008)

    Article  MATH  Google Scholar 

  45. Pinel, F., Pecero, J.E., Bouvry, P., Khan, S.U.: A two-phase heuristic for the scheduling of independent tasks on computational grids. In: Proc. of the ACM/IEEE/IFIP International Conference on High Performance Computing and Simulation (HPCS), Istanbul, Turkey, July 2011, pp. 471–477 (2011)

    Google Scholar 

  46. Pinel, F., Pecero, J.E., Khan, S.U., Bouvry, P.: Utilizing GPUs to solve large instances of the tasks mapping problem. In: Proc. of the International Research Workshop on Advanced High Performance Computing Systems, Cetraro, Italy, June 2011

    Google Scholar 

  47. Pinel, F., Dorronsoro, B., Pecero, J.E., Bouvry, P., Khan, S.U.: A two-phase heuristic for the energy-efficient scheduling of independent tasks on computational grids. Clust. Comput.. doi:10.1007/s10586-012-0207-x (2012).

    Google Scholar 

  48. Qureshi, K., Rehman, A., Manuel, P.: Enhanced GridSim architecture with load balancing. J. Supercomput. (2010). doi:10.1007/s11227-010-0402-6

    Google Scholar 

  49. Rus, P., Stok, B., Mole, N.: Parallel computing with load balancing on heterogeneous distributed systems. Adv. Eng. Softw. 34(4), 185–201 (2003)

    Article  MATH  Google Scholar 

  50. Theoharakis, V., Guerin, R.: SONET OD-12 interface for variable length packets. In: Proc. of the 2nd International Conference on Computer Communication and Networks (1993)

    Google Scholar 

  51. Transmission control protocol, RFC 793. http://www.faqs.org/rfcs/rfc793.html. Accessed Apr 2013

  52. Traw, C.B.S., Smith, J.: Striping within the network subsystem. IEEE Netw. 9(4), 22–32 (1995)

    Article  Google Scholar 

  53. Tudruj, M., Borkowski, J., Koponski, D.J.: Load balancing with migration based on synchronizers in PS-GRADE graphical tool. In: Proc. of the 4th International Symposium on Parallel and Distributed Computing, July 2005, pp. 105–112 (2005)

    Chapter  Google Scholar 

  54. Valentini, G.L., Lassonde, W., Khan, S.U., Min-Allah, N., Madani, S.A., Li, J., Zhang, L., Wang, L., Ghani, N., Kolodziej, J., Li, H., Zomaya, A.Y., Xu, C.-Z., Balaji, P., Vishnu, A., Pinel, F., Pecero, J.E., Kliazovich, D., Bouvry, P.: An overview of energy efficiency techniques in cluster computing systems. Clust. Comput. 16(1), 3–15 (2013)

    Article  Google Scholar 

  55. Yagoubi, B., Medebber, M.: A load balancing model for grid environment. In: Proc. of the 22nd International Symposium on Computer and Information Sciences, Turkey, pp. 1–7 (2007)

    Google Scholar 

  56. Zamanifar, K., Nematbakhsh, N., Sadjady, R.S.: A new load balancing algorithm in parallel computing. In: Proc. of the 2nd International Conference on Communication Software and Networks (ICCSN), pp. 449–453 (2010)

    Google Scholar 

  57. Zhang, Q., Li, Z.: Data transfer based on multiple replicas in the grid environment. In: Proc. of the 5th Annual ChinaGrid Conference, Guangzhou, China, July 2010, pp. 240–244 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information Technology, United Arab Emirates University, P.O. Box 17551, Al Ain, UAE

    Nader Mohamed

  2. Middleware Technologies Lab., Aali, Bahrain

    Jameela Al-Jaroodi

  3. Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA

    Hong Jiang

Authors
  1. Nader Mohamed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jameela Al-Jaroodi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nader Mohamed.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mohamed, N., Al-Jaroodi, J. & Jiang, H. DDOps: dual-direction operations for load balancing on non-dedicated heterogeneous distributed systems. Cluster Comput 17, 503–528 (2014). https://doi.org/10.1007/s10586-013-0294-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-013-0294-3

Keywords

Search

Navigation