Multi-agent model predictive control based on resource allocation coordination for a class of hybrid systems with limited information sharing
Introduction
Multi-agent systems, like transportation systems, manufacturing systems, power systems, financial systems, are composed of multiple subsystems with interactions (Kantamneni et al., 2015). Multi-agent systems research is facing a variety of challenges (Leitão, 2009), of which a crucial one is to design mechanisms for coordinating agents that have limited information sharing with each other in order to protect confidential information of local subsystems while at the same time still aiming for global performance (Dutta et al., 2005). Typical global control goals for multi-agent systems involve synchronizing motions of agents, maximizing resource utility, and minimizing control costs. In order to achieve globally satisfactory performance, given limited information of other subsystems, the agents need to assist each other to make better decisions about their actions (Cao et al., 2013). Thanks to its straightforward design procedure, where hard system constraints are incorporated directly as inequalities in the formulation of the control problem, model predictive control has shown to be a promising control strategy for multi-agent systems (Camponogara et al., 2002, Dunbar and Murray, 2006, Negenborn et al., 2008, Scattolini, 2009).
However, the cooperation among agents is made much more difficult when the individual agents have to regulate hybrid subsystems (Christofides et al., 2013) that contain both continuous components and discrete components, such as switches and overrides. In fact, this will result in having to solve mixed-integer programming problems in a distributed way, for which there has not yet been a universally successful algorithm (Frick et al., 2015). Moreover, many system-theoretic concepts and control strategies, such as model predictive control and Artificial Intelligence based control (Olaru et al., 2004, Cai et al., 2014), still require further examination and research in this setting (Mayne et al., 2000).
In this paper, we focus on a class of hybrid systems that are governed by discrete inputs and that are subject to global hard constraints. In particular, each subsystem is characterized by a convex objective function and a strictly increasing constraint function with respect to the local control variable. Besides, each subsystem only shares limited information with the external environment. Actually, such hybrid systems are ubiquitous, and an important example are a group of systems with on/off switches sharing a given amount of resources. More specifically, concrete real-life examples include the charging of a fleet of electric vehicles sharing a given power level provided by the grid, and the operation of a number of appliances sharing a given amount of energy in smart buildings. We aim to develop a multi-agent model predictive control method for such a class of hybrid systems based on a distributed resource allocation coordination algorithm.
A resource allocation is a plan for using the available resources to achieve goals for the future. In principle, such planning may be done by centrally scheduling the actions of the systems that require resources (Huang et al., 2013). However, for reasons of scalability and fast computation, it will not be tractable to schedule the actions of a large number of systems in a centralized way. Actually, the scheduling of the actions of the systems that require resources can be done in a distributed way based on the primal decomposition (Boyd and Vandenberghe, 2004) of the overall problem. More specifically, in primal decomposition, which is naturally applicable to resource allocation scenarios, the allocation of resources can be represented by auxiliary variables and these variables are optimized using a master problem (Palomar and Chiang, 2006). A resource allocation coordination algorithm that is based on the primal decomposition of the overall problem, has already been developed for continuous optimization problems with global capacity constraints by Cohen (1978).
In fact, a multi-agent control method based on the primal decomposition of the overall control problem will always guarantee the global feasibility of local control decisions. However, since all the control variables are discrete in the control setting considered in the paper, issues such as oscillatory behavior of the discrete decision variables could arise if the resource allocation coordination algorithm is applied directly. As a result, the global optimality of the algorithm cannot be guaranteed anymore.
In this paper, a smart mechanism based on the branch-and-bound paradigm (Lawler and Wood, 1966) is developed to improve the solution found when using the resource allocation coordination algorithm only. More specifically, this is achieved by building the search tree according to the outcome of the resource allocation coordination algorithm at each node and by returning the best solution found when the overall method stops.
In the literature, a mechanism has been proposed by Bourdais et al. (2012) to deal with the oscillatory behavior of the discrete decision variables. That mechanism is also based on the branch-and-bound paradigm but it uses a distributed algorithm based on the dual decomposition of the overall problem. Since in the dual decomposition approach constraints are relaxed and accounted for in the objective by using penalties for violations, it cannot be guaranteed that the constraints are always satisfied during iterations. Moreover, a general framework of embedded optimization based on the branch-and-bound paradigm has been presented by Frick et al. (2015) for model predictive control of hybrid systems, which includes the class of hybrid systems considered in this paper. However, that paper focuses on building a problem-specific branch-and-bound search tree by pre-processing heuristics. So, to the authors’ best knowledge, an online and problem-independent algorithm for the control of the class of hybrid systems considered here has not yet been proposed in the literature.
In our previous work (Luo et al., 2015a), we have integrated the resource allocation coordination method into the branch-and-bound paradigm. However, in (Luo et al., 2015a), we assumed the solver to have full information of the overall problem, and we did not consider protecting the confidential information of the local subproblems. In the contrast, the main contribution of this paper consists in reducing information sharing among local control agents, which helps protecting the confidential information of the local subproblems.
With respect to the literature, the multi-agent control method proposed in this paper only requires limited information sharing among local control agents. In addition, it guarantees the global feasibility of local control decisions and it is also able to efficiently search the overall solution space online by making use of the outcome of the resource allocation coordination algorithm at each node of the tree.
This paper is organized as follows. In Section 2, the considered class of hybrid systems with subsystems governed by discrete inputs and subject to global hard constraints is formalized. In Section 3, the resource allocation coordination algorithm based on a projected subgradient method is presented. In Section 4, we present the overall proposed multi-agent model predictive control method. In Section 5, we consider charging control of a fleet of electric vehicles as an application example of the proposed method and assess the performance of the proposed method in a simulation study. Section 6 summarizes the results of this paper and presents some ideas for future work.
Section snippets
Model predictive control for a class of hybrid systems
In this section, we focus on the control problem formulation of hybrid systems governed by discrete inputs and subject to global hard constraints. Assume that a large-scale hybrid system consists of N subsystems such that:
- •
Each subsystem is controlled by a control agent.
- •
Each control agent has a dynamical model of its subsystem.
- •
Each control agent has to solve its local problem.
- •
Each agent does not have any information of the models and the local control problems of other subsystems.
- •
Subsystems
Resource allocation coordination
Before presenting the overall multi-agent model predictive control method, in the section, we first describe the resource allocation coordination algorithm on which the overall control method will be based. In the resource allocation coordination algorithm, the maximum amount of resources allocated to each subproblem is represented by an auxiliary variable and then the coordination is achieved by solving a master problem that optimizes all the auxiliary variables (Palomar and Chiang, 2006).
Multi-agent model predictive control method based on resource allocation coordination
Although the values of the control decision variables may oscillate when the resource allocation coordination algorithm is applied to the combined overall control problem (5), the oscillations of the values of the control decision variables can be used as a guideline for branching the overall solution space. In this section, we develop a multi-agent control method for the combined overall control problem (5) by integrating the resource allocation coordination algorithm into a search tree
Charging control of electric vehicles
As an application example of the developed multi-agent model predictive control method, in this section we address the charging control of a fleet of electric vehicles under constrained grid conditions.
Due to their higher energy efficiency and lower emission of pollutants, electric vehicles are used more and more. Charging this increasing number of electric vehicles will inevitably cause additional load to the electrical power distribution grid (Hu et al., 2015, Fernandez et al., 2011).
Conclusions and future work
We have considered multi-agent model predictive control for a class of hybrid systems governed by discrete inputs and subject to global hard constraints where each subsystem has a local convex objective function and a strictly increasing constraint function. We focused on the scenario where each subsystem only shares limited information with the external environment, and we developed a novel multi-agent model predictive control method by integrating a distributed resource allocation
Acknowledgments
This research was supported by Ammodo via the Van Gogh project VGP.14/47 and by the China Scholarship Council under Grant 201207090001.
References (34)
- et al.
Distributed model predictive control: a tutorial review and future research directions
Comput. Chem. Eng.
(2013) - et al.
Distributed receding horizon control for multi-vehicle formation stabilization
Automatica
(2006) - et al.
Embedded optimization for mixed logical dynamical systems
Comput. Chem. Eng.
(2015) - et al.
A multi-agent system for distribution grid congestion management with electric vehicles
Eng. Appl. Artif. Intell.
(2015) - et al.
An adaptive resource management scheme in cloud computing
Eng. Appl. Artif. Intell.
(2013) - et al.
Survey of multi-agent systems for microgrid control
Eng. Appl. Artif. Intell.
(2015) Agent-based distributed manufacturing control: a state-of-the-art survey
Eng. Appl. Artif. Intell.
(2009)- et al.
Constrained model predictive control: stability and optimality
Automatica
(2000) - et al.
Multi-agent model predictive control for transportation networks: serial versus parallel schemes
Eng. Appl. Artif. Intell.
(2008) Architectures for distributed and hierarchical model predictive control-a review
J. Process Control
(2009)
A decomposition method for large scale MILPs, with performance guarantees and a power system application
Automatica
Nonlinear Programming
Convex Optimization
Distributed model predictive control
IEEE Control Syst. Mag.
An overview of recent progress in the study of distributed multi-agent coordination
IEEE Trans. Ind. Inform.
Cited by (12)
Swarm control based on artificial potential field method with predicted state and input threshold
2023, Engineering Applications of Artificial IntelligenceIntegrated condition-based track maintenance planning and crew scheduling of railway networks
2019, Transportation Research Part C: Emerging TechnologiesCitation Excerpt :Although the solutions of both approaches are suboptimal, numerical experiments show that the loss of optimality is small for the corresponding application. However, unlike the real-world problems mentioned in Luo et al. (2017) and Mendes et al. (2017), where some restrictions on the information exchange prohibit the implementation of a centralized optimization scheme, we adopt a distributed optimization scheme purely out of computational concerns. We apply the MLD framework to transform the hybrid deterioration dynamics, resulting in an MILP problem to be solved at each time step.