Abstract
In this paper, we study the problem of task reallocation for load-balancing in distributed data processing models that tackle vast amount of data. We propose a strategy based on cooperative agents used to optimize the rescheduling of tasks in multiple jobs which must be executed as soon as possible. It allows agents to determine locally the next tasks to process, to delegate, possibly to swap according to their knowledge, their own belief base and their peer modelling. The novelty lies in the ability of agents to identify opportunities and bottleneck agents, and afterwards to reassign some bundles of tasks thanks to concurrent bilateral negotiations. The strategy adopted by the agents allows to warrant a continuous improvement of the flowtime. Our experimentation reveals that our strategy reaches a flowtime which is better than the one reached by a DCOP resolution, close to the one reached by the classical heuristic approach, and significantly reduces the rescheduling time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baert, Q., Caron, A.C., Morge, M., Routier, J.C., Stathis, K.: An adaptive multi-agent system for task reallocation in a MapReduce job. J. Parallel Distrib. Comput. 153, 75–88 (2021). https://doi.org/10.1016/j.jpdc.2021.03.008
Beauprez, E., Caron, A.C., Morge, M., Routier, J.C.: A multi-agent negotiation strategy for reducing the flowtime. In: Proceedings of 13th International Conference on Agents and Artificial Intelligence (ICAART), vol. 1, pp. 58–68, February 2021
Beauprez, E., Morge, M.: Scala implementation of the extended multi-agents situated task allocation (2020). https://gitlab.univ-lille.fr/maxime.morge/smastaplus
Beynier, A., Charpillet, F., Szer, D., Mouaddib, A.I.: DEC-MDP / DEC-POMDP. In: Olivier Buffet, O.S. (ed.) Markov Decision Processes in Artificial Intelligence, pp. 277–313. Wiley-ISTE (2010)
Bruno, J., Coffman, E.G., Jr., Sethi, R.: Scheduling independent tasks to reduce mean finishing time. Commun. ACM 17(7), 382–387 (1974)
Chen, B., Potts, C.N., Woeginger, G.J.: A review of machine scheduling: complexity, algorithms and approximability. In: Du, D.Z., Pardalos, P.M. (eds.) Handbook of Combinatorial Optimization, pp. 1493–1641. Springer, Boston (1998). https://doi.org/10.1007/978-1-4613-0303-9_25
Choi, H.L., Brunet, L., How, J.P.: Consensus-based decentralized auctions for robust task allocation. IEEE Trans. Rob. 25(4), 912–926 (2009)
Conway, R., Maxwell, W., Miller, L.: Theory of Scheduling. Addison- Wesley, Reading (1967)
Fioretto, F., Pontelli, E., Yeoh, W.: Distributed constraint optimization problems and applications: a survey. J. Artif. Intell. Res. 61, 623–698 (2018)
Fulgham, B., Gouy, I.: The Computer Language Benchmarks Game (2021). https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html
Horn, W.: Minimizing average flow time with parallel machines. Oper. Res. 21(3), 846–847 (1973)
Jiang, Y.: A survey of task allocation and load balancing in distributed systems. IEEE Trans. Parallel Distrib. Syst. 27(2), 585–599 (2016)
Kuhn, H.W.: The Hungarian method for the assignment problem. Naval Res. Logistics Q. 2(1–2), 83–97 (1955)
Lesser, V., et al.: Evolution of the GPGP/TAEMS domain-independent coordination framework. Auton. Agent. Multi-Agent Syst. 9(1–2), 87–143 (2004)
Li, S., Negenborn, R.R., Lodewijks, G.: A distributed constraint optimization approach for vessel rotation planning. In: González-Ramírez, R.G., Schulte, F., Voß, S., Ceroni Díaz, J.A. (eds.) ICCL 2014. LNCS, vol. 8760, pp. 61–80. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11421-7_5
Lightbend: Akka is the implementation of the actor model on the JVM (2020). http://akka.io
Maheswaran, R.T., Pearce, J.P., Tambe, M.: Distributed algorithms for DCOP: a graphical-game-based approach. In: Bader, D.A., Khokhar, A.A. (eds.) Proceedings of the ISCA 17th International Conference on Parallel and Distributed Computing Systems, The Canterbury Hotel, San Francisco, California, USA, 15–17 September 2004, pp. 432–439. ISCA (2004)
Rubinstein, A.: Perfect equilibrium in a bargaining model. Econometrica 50(1), 97–102 (1982)
Rust, P., Picard, G.: pyDCOP is a python library for Distributed Constraints Optimization (2021). https://github.com/Orange-OpenSource/pyDcop
Schaerf, A., Shoham, Y., Tennenholtz, M.: Adaptive load balancing: a study in multi-agent learning. J. Artif. Intell. Res. 2, 475–500 (1995)
Shehory, O., Kraus, S.: Methods for task allocation via agent coalition formation. Artif. Intell. 101(1–2), 165–200 (1998)
Turner, J., Meng, Q., Schaefer, G., Soltoggio, A.: Distributed strategy adaptation with a prediction function in multi-agent task allocation. In: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), pp. 739–747 (2018)
Wellman, M.P.: A market-oriented programming environment and its application to distributed multicommodity flow problems. J. Artif. Intell. Res. 1, 1–23 (1993)
Zaharia, M., et al.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI), San Jose, CA, USA, pp. 15–28 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Beauprez, E., Caron, A.C., Morge, M., Routier, JC. (2022). Task Bundle Delegation for Reducing the Flowtime. In: Rocha, A.P., Steels, L., van den Herik, J. (eds) Agents and Artificial Intelligence. ICAART 2021. Lecture Notes in Computer Science(), vol 13251. Springer, Cham. https://doi.org/10.1007/978-3-031-10161-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-10161-8_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10160-1
Online ISBN: 978-3-031-10161-8
eBook Packages: Computer ScienceComputer Science (R0)