Abstract
This work deals with the problem of navigation while avoiding detection by a mobile adversary, featuring adversarial modeling. In this problem, an evading agent is placed on a graph, where one or more nodes are defined as safehouses. The agent’s goal is to find a path from its current location to a safehouse, while minimizing the probability of meeting a mobile adversarial agent at a node along its path (i.e., being captured). We examine several models of this problem, where each one has different assumptions on what the agents know about their opponent, all using a framework for computing node utility, introduced herein. Using risk attitudes for computing the utility values, their impact on the constructed strategies is analyzed both theoretically and empirically. Furthermore, we allow the agents to use information gained along their movement, in order to efficiently update their motion strategies on-the-fly. Theoretical and empirical analysis shows the importance of using this information and these on-the-fly strategy updates.
Similar content being viewed by others
References
Adler, M., Räcke, H., Sivadasan, N., Sohler, C., Vöcking, B.: Randomized pursuit-evasion in graphs. Comb. Probab. Comput. 12(03), 225–244 (2003)
Aigner, M., Fromme, M.: A game of cops and robbers. Discret. Appl. Math. 8(1), 1–12 (1984)
Alexopoulos, A., Schmidt, T., Badreddin, E.: Cooperative pursue in pursuit-evasion games with unmanned aerial vehicles. In: Proceedings of the International Conference on Intelligent Robots and Systems (IROS). IEEE (2015)
Alpern, S., Fokkink, R., Gal, S., Timmer, M.: On search games that include ambush. SIAM J. Control. Optim. 51(6), 4544–4556 (2013)
Alspach, B.: Searching and sweeping graphs: a brief survey. Le matematiche 59 (1, 2), 5–37 (2006)
Barrett, S., Stone, P., Kraus, S.: Empirical evaluation of ad hoc teamwork in the pursuit domain. In: Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS), vol. 2, pp. 567–574. International Foundation for Autonomous Agents and Multiagent Systems (2011)
Basilico, N., De Nittis, G., Gatti, N.: A security game combining patrolling and alarm–triggered responses under spatial and detection uncertainties. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 397–403. AAAI Press (2016)
Basilico, N., Gatti, N., Amigoni, F.: Leader-follower strategies for robotic patrolling in environments with arbitrary topologies. In: Proceedings of the International Conference on Autonomous Agents and Multiagent Systems(AAMAS), vol. 1, pp. 57–64. International Foundation for Autonomous Agents and Multiagent Systems (2009)
Bhadauria, D., Klein, K., Isler, V., Suri, S.: Capturing an evader in polygonal environments with obstacles: The full visibility case. The International Journal of Robotics Research 31(10), 1176–1189 (2012)
Bishop, C.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, New York (2007)
Boidot, E., Marzuoli, A., Feron, E.: Optimal navigation policy for an autonomous agent operating in adversarial environments. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3154–3160. IEEE (2016)
Borie, R.B., Tovey, C.A., Koenig, S.: Algorithms and complexity results for pursuit-evasion problems. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), vol. 9, pp. 59–66 (2009)
Brémaud, P.: Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues, vol. 31. Springer, New Yoork (2013)
Chen, H., Zhang, F.: The expected hitting times for finite markov chains. Linear Algebra Appl. 428(11-12), 2730–2749 (2008)
Chung, T.H., Hollinger, G.A., Isler, V.: Search and pursuit-evasion in mobile robotics. Auton. Robot. 31(4), 299–316 (2011)
Daskalakis, C., Goldberg, P.W., Papadimitriou, C.H.: The complexity of computing a nash equilibrium. SIAM J. Comput. 39(1), 195–259 (2009)
De Alfaro, L., Henzinger, T.A., Kupferman, O.: Concurrent reachability games. Theor. Comput. Sci. 386(3), 188–217 (2007)
Dereniowski, D., Dyer, D., Tifenbach, R.M., Yang, B.: Zero-visibility cops and robber game on a graph. In: Frontiers in Algorithmics and Algorithmic Aspects in Information and Management, pp. 175–186. Springer (2013)
Foderaro, G., Raju, V., Ferrari, S.: A cell decomposition approach to online evasive path planning and the video game Ms. Pac-Man. In: IEEE International Symposium on Intelligent Control (ISIC), pp. 191–197 (2011)
Fomin, F.V., Thilikos, D.M.: An annotated bibliography on guaranteed graph searching. Theor. Comput. Sci. 399(3), 236–245 (2008)
Geraerts, R., Schager, E.: Stealth-based path planning using corridor maps. In: Proceedings of Computer Animation and Social Agents (CASA) (2010)
Gibbs, A., Su, F.: On choosing and bounding probability metrics. Int. Stat. Rev. 70(3), 419–435 (2002)
Gmytrasiewicz, P.J., Doshi, P.: A framework for sequential planning in multi-agent settings. J. Artif. Intell. Res. (JAIR) 24, 49–79 (2005)
Göbel, F., Jagers, A.: Random walks on graphs. Stochastic Processes and their Applications 2(4), 311–336 (1974)
Guibas, L.J., Latombe, J.C., LaValle, S.M., Lin, D., Motwani, R.: A visibility-based pursuit-evasion problem. Int. J. Comput. Geom. Appl. 9(4 & 5), 471–493 (1999)
Huang, H., Ding, J., Zhang, W., Tomlin, C.J.: Automation-assisted capture-the-flag: a differential game approach. IEEE Trans. Control Syst. Technol. 23 (3), 1014–1028 (2015)
Jain, M., Conitzer, V., Tambe, M.: Security scheduling for real-world networks. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 215–222 (2013)
Karlin, S.: A First Course in Stochastic Processes. Academic Press, New York (2014)
Latombe, J.C.: Robot Motion Planning. Kluwer Academic Publishers, Boston (1991)
Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the Eleventh International Conference on Machine Learning, vol. 157, pp. 157–163 (1994)
Lubiw, A., Vosoughpour, H.: Visibility graphs, dismantlability, and the cops and robbers game. In: 26th Canadian conference on computational geometry (2014)
Marzouqi, M., Jarvis, R.A.: Covert path planning for autonomous robot navigation in known environments. In: Proceedings of the Australasian Conference on Robotics and Automation, Citeseer, Brisbane (2003)
Marzouqi, M., Jarvis, R.A.: Covert robotics: covert path planning in unknown environments. In: Proceedings of the Australasian Conference on Robotics and Automation, Citeseer, Brisbane (2003)
Maskin, E., Tirole, J.: Markov perfect equilibrium, i: Observable actions. J. Econ. Theory 100(2), 191–219 (2001)
McCarthy, S., Tambe, M., Kiekintveld, C., Gore, M.L., Killion, A.: Preventing illegal logging: simultaneous optimization of resource teams and tactics for security. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 3880–3886 (2016)
Nearchou, A.C.: Path planning of a mobile robot using genetic heuristics. Robotica 16(05), 575–588 (1998)
Nowakowski, R., Winkler, P.: Vertex-to-vertex pursuit in a graph. Discret. Math. 43(2), 235–239 (1983)
Örgen, P., Winstrand, M.: Minimizing mission risk in fuel constrained uav path planning. AIAA Journal of Guidance Control, and Dynamics 31(5), 1497–1500 (2008)
Petres, C., Pailhas, Y., Patron, P., Petillot, Y., Evans, J., Lane, D.: Path planning for autonomous underwater vehicles. IEEE Trans. Robot. 23(2), 331–341 (2007)
Raghavan, T.: Zero-sum two-person games. Handbook of Game Theory with Economic Applications 2, 735–768 (1994)
Sgall, J.: Solution of david gale’s lion and man problem. Theor. Comput. Sci. 259(1-2), 663–670 (2001)
Shoham, Y., Leyton-Brown, K.: Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge (2008)
Stentz, A.: Optimal and efficient path planning for partially-known environments. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 3310–3317 (1994)
Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning. MIT Press, Cambridge (1998)
Tews, A., Matarić, M.J., Sukhatme, G.S.: Avoiding detection in a dynamic environment. In: Proceedings of the International Conference on Intelligent Robots and Systems (IROS), vol. 4, pp. 3773–3778 (2004)
Tews, A., Sukhatme, G.S., Matarić, M.J.: A multi-robot approach to stealthy navigation in the presence of an observer. In: Proceedings of the International Conference on Robotics and Automation, pp. 2379–2385 (2004)
Yehoshua, R., Agmon, N.: Adversarial modeling in the robotic coverage problem. In: Proceedings of International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS) (2015)
Zhang, Y., An, B., Tran-Thanh, L., Wang, Z., Gan, J., Jennings, N.R.: Optimal escape interdiction on transportation networks. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (2017)
Acknowledgments
This paper was supported in part by ISF grant 1337/15.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
We shall now introduce the full proof of Theorem 4:
Theorem 4
Denote the actual probability distribution over C’s location at time t as \(P_{\mathrm {C}}^{(t)}\) (i.e., \(P_{\mathrm {C}}^{(t)}\) is 1 for C’s location and 0 for any other node). \(\tilde {P}_{\mathrm {C}}^{(t)}\) denotes R’s estimation for \(P_{\mathrm {C}}^{(t)}\) before updating it with the information received from \(V_{visible}(v_{\mathrm {R}}^t)\), while \(\hat {P}_{\mathrm {C}}^{(t)}\)denotes \(\tilde {P}_{\mathrm {C}}^{(t)}\) after being updated. Incorporating the information received from the visibility edges for updating \(\hat {P}_{\mathrm {C}}^{(t)}\), improves the accuracy of this estimation. Namely, \(H(\tilde {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)}) \geq H(\hat {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)})\).
Proof
As stated in Section 6.2, if \({\sum }_{v \in V_{visible}(v_{\mathrm {R}}^t)} \tilde {P}_{\mathrm {C}}^{(t)}[v] = 0\) then \(\hat {P}_{\mathrm {C}}^{(t)} = \tilde {P}_{\mathrm {C}}^{(t)}\). Hence, \(H(\tilde {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)}) = H(\hat {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)})\).
Now, assume \({\sum }_{v \in V_{visible}(v_{\mathrm {R}}^t)} \tilde {P}_{\mathrm {C}}^{(t)}[v] > 0\) (a sum of probabilities is either 0 or positive). If C has been observed (\(v_{\mathrm {C}}^{t} \in V_{visible}(v_{\mathrm {R}}^t)\)), then \(\hat {P}_{\mathrm {C}}^{(t)}\) = \(P_{\mathrm {C}}^{(t)}\).
Otherwise, \(v_{\mathrm {C}}^{t} \notin V_{visible}(v_{\mathrm {R}}^t)\). Let us compute the Hellinger distance between the actual distribution over C’s location to R’ estimated distribution prior being updated, \(H\left (\tilde {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)}\right )\):
C resides at \(v_{\mathrm {C}}^{t}\), hence \(\forall v \in V \setminus \{v_{\mathrm {C}}^{t}\}\), \(P_{\mathrm {C}}^{t}[v] = 0\), \(P_{\mathrm {C}}^{t}[v_{\mathrm {C}}^{t}] = 1\). Therefore:
Now, we shall compute the Hellinger distance between the actual distribution over C’s location to R’ estimated distribution after being updated, \(H\left (\hat {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)}\right )\):
C resides at \(v_{\mathrm {C}}^{t}\) and \(v_{\mathrm {C}}^{t} \notin V_{visible}(v_{\mathrm {R}}^t)\), hence:
Therefore, we obtain:
If \(\tilde {P}_{\mathrm {C}}^{(t)}[v_{\mathrm {C}}^{t}] = 0\) then \(\hat {P}_{\mathrm {C}}^{(t)}[v_{\mathrm {C}}^{t}] = \tilde {P}_{\mathrm {C}}^{(t)}[v_{\mathrm {C}}^{t}]\) and \(H(\tilde {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)}) = H(\hat {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)})\). Otherwise:
In conclusion, we obtain that \(H(\tilde {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)}) \geq H(\hat {P}_{\mathrm {C}}^{(t)},P_{\mathrm {C}}^{(t)})\), hence R’s estimated probability distribution over C’s location is closer to \(P_{\mathrm {C}}^{(t)}\) after incorporating the information gained from the visibility edges. □
Proving Theorem 4 for C is similar.
Rights and permissions
About this article
Cite this article
Keidar, O., Agmon, N. Safe navigation in adversarial environments. Ann Math Artif Intell 83, 121–164 (2018). https://doi.org/10.1007/s10472-018-9591-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10472-018-9591-0