Abstract
We study a class of deterministic two-player nonzero-sum differential games where one player uses piecewise-continuous controls to affect the continuously evolving state, while the other player uses impulse controls at certain discrete instants of time to shift the state from one level to another. The state measurements are made at some given instants of time, and players determine their strategies using the last measured state value. We provide necessary and sufficient conditions for the existence of sampled-data Nash equilibrium for a general class of differential games with impulse controls. We specialize our results to a scalar linear-quadratic differential game and show that the equilibrium impulse timing can be obtained by determining a fixed point of a Riccati-like system of differential equations with jumps coupled with a system of nonlinear equality constraints. By reformulating the problem as a constrained nonlinear optimization problem, we compute the equilibrium timing, and level of impulses. We find that the equilibrium piecewise continuous control and impulse control are linear functions of the last measured state value. Using a numerical example, we illustrate our results.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In the paper, the necessary and sufficient conditions are obtained for any finite equilibrium number of impulses. However, in the case of linear-quadratic games, we fix the number of impulses in each sampling interval (see Sect. 4).
[25] obtained the Hamiltonian continuity condition for a general class of linear-quadratic differential games under the open-loop information structure.
If the solution y(t) of a nonlinear ordinary differential equation \({\dot{y}} = f(y,t), y(0)=y_0\) becomes unbounded as \(t \rightarrow t_e\) where \(t_e<\infty \), then \(t_e\) is called the finite escape time [19, 24]. In [19], it is shown that the Riccati differential equation \({\dot{y}}(t)=sy(t)^2+2ay(t)+h\), \(y(T)=q_T\) has a solution for every \(T>0\) if \(d=a^2-hs\ge 0\) and \(q_T> \frac{a+\sqrt{d}}{s}\).
References
Aïd, R., Basei, M., Callegaro, G., Campi, L., Vargiolu, T.: Nonzero-sum stochastic differential games with impulse controls: a verification theorem with applications. Math. Oper. Res. 45(1), 205–232 (2020)
Aïd, R., Campi, L., Li, L., Ludkovski, M.: An impulse-regime switching game model of vertical competition. Dyn. Games Appl. (2021). https://doi.org/10.1007/s13235-021-00381-4. (ePub ahead of print, March 22)
Başar, T.: On the existence and uniqueness of closed-loop sampled-data Nash controls in linear-quadratic stochastic differential games. In: Iracki, K., Malanowski, K., Walukiewicz, S. (eds.) Optimization Techniques, pp. 193–203. Springer, Berlin Heidelberg, Berlin, Heidelberg (1980)
Başar, T.: Time consistency and robustness of equilibria in noncooperative dynamic games. In: der Ploeg, F.V., de Zeeuw, A. (eds.) Dynamic Policy Games in Economics, pp. 9–54. Springer International Publishing, North-Holland (1989)
Başar, T.: Optimum H\(\infty \) designs under sampled state measurements. Syst. Control Lett. 16(6), 399–409 (1991)
Başar, T.: Minimax control of switching systems under sampling. Syst. Control Lett. 25(5), 315–325 (1995)
Başar, T., Olsder, G.: Dynamic Noncooperative Game Theory, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia, PA (1999)
Başar, T., Haurie, A., Zaccour, G.: Nonzero-sum differential games. In: Başar, T., Zaccour, G. (eds.) Handbook of Dynamic Game Theory, pp. 61–110. Springer International Publishing, Cham (2018)
Basei, M., Cao, H., Guo, X.: Nonzero-sum stochastic games and mean-field games with impulse controls. Math. Oper. Res. (2021) (forthcoming)
Blaquière, A.: Differential games with piece-wise continuous trajectories. In: Hagedorn, P., Knobloch, H.W., Olsder, G.J. (eds.) Differential Games and Applications, pp. 34–69. Springer, Berlin (1977)
Blaquière, A.: Necessary and sufficient conditions for optimal strategies in impulsive control and application. In: Aoki, M., Morzzolla, A. (eds.) New Trends in Dynamic System Theory and Economics, pp. 183–213. Academic Press, New York (1977)
Büskens, C., Maurer, H.: Sqp-methods for solving optimal control problems with control and state constraints: adjoint variables, sensitivity analysis and real-time control. J. Comput. Appl. Math. 120(1), 85–108 (2000)
Byrd, R.H., Hribar, M.E., Nocedal, J.: An interior point algorithm for large-scale nonlinear programming. SIAM J. Optim. 9(4), 877–900 (1999)
Campi, L., De Santis, D.: Nonzero-sum stochastic differential games between an impulse controller and a stopper. J. Optim. Theory Appl. 186(2), 688–724 (2020)
Chahim, M., Hartl, R.F., Kort, P.M.: A tutorial on the deterministic impulse control maximum principle: necessary and sufficient optimality conditions. Eur. J. Oper. Res. 219(1), 18–26 (2012)
Drăgan, V., Ivanov, I.G., Popa, I.L.: Stochastic linear quadratic differential games in a state feedback setting with sampled measurements. Syst. Control Lett. 134, 104,563 (2019)
Dockner, E.J., Jorgensen, S., Long, N.V., Sorger, G.: Differential Games in Economics and Management Science. Cambridge University Press, Cambridge (2000)
El Farouq, N., Barles, G., Bernhard, P.: Deterministic minimax impulse control. Appl. Math. Optim. 61(3), 353–378 (2010)
Engwerda, J.C.: LQ Dynamic Optimization and Differential Games. Wiley, New York (2005)
Ferrari, G., Koch, T.: On a strategic model of pollution control. Ann. Oper. Res. 275(2), 297–319 (2019)
Geering, H.: Continuous-time optimal control theory for cost functionals including discrete state penalty terms. IEEE Trans. Autom. Control 21(6), 866–869 (1976)
Haurie, A., Krawczyk, J.B., Zaccour, G.: Games and Dynamic Games. World Scientific, Singapore (2012)
Imer, O.C., Başar, T.: To measure or to control: optimal control with scheduled measurements and controls. In 2006 American Control Conference, pp. 1003–1008 (2006)
Jacobson, D.H.: Extensions of Linear-Quadratic Control, Optimization and Matrix Theory. Academic Press, London-New York (1977)
Sadana, U., Reddy, P.V., Zaccour, G.: Nash equilibria in nonzero-sum differential games with impulse control. Eur. J. Oper. Res. 295(2), 792–805 (2021)
Simaan, M., Cruz, J., Jr.: Sampled-data Nash controls in non-zero-sum differential games. Int. J. Control 17(6), 1201–1209 (1973)
Acknowledgements
The authors would like to thank the Editor and the three anonymous reviewers for their careful reading and helpful suggestions on an earlier version of the manuscript. The first author’s research is supported by the FRQNT International Internship Program (278894). He also thanks the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, USA, for its hospitality. The second author’s research is supported by SERB, Government of India, grant MTR/2019/000771. Research of the third author was supported in part by the Air Force Office of Scientific Research (AFOSR) Grant FA9550-19-1-0353.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Hélène Frankowska.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sadana, U., Reddy, P.V., Başar, T. et al. Sampled-Data Nash Equilibria in Differential Games with Impulse Controls. J Optim Theory Appl 190, 999–1022 (2021). https://doi.org/10.1007/s10957-021-01920-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-021-01920-0