Skip to main content
SpringerLink
Log in

Optimizing Evasive Strategies for an Evader with Imperfect Vision Capacity

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The multiagent pursuit-evasion problem has attracted considerable interest during recent years, and a general assumption is that the evader has perfect vision capacity. However, in the real world, the vision capacity of the evader is always imperfect, and it may have noisy observation within its limited field of view. Such an imperfect vision capacity makes the evader sense incomplete and inaccurate information from the environment, and thus, the evader will achieve suboptimal decisions. To address this challenge, we decompose this problem into two subproblems: 1) optimizing evasive strategies with a limited field of view, and 2) optimizing evasive strategies with noisy observation. For the evader with a limited field of view, we propose a memory-based ‘worst case’ algorithm, the idea of which is to store the locations of the pursuers seen before and estimate the possible region of the pursuers outside the sight of the evader. For the evader with noisy observation, we propose a value-based reinforcement learning algorithm that trains the evader offline and applies the learned strategy to the actual environment, aiming at reducing the impact of uncertainty created by inaccurate information. Furthermore, we combine and make a trade-off between the above two algorithms and propose a memory-based reinforcement learning algorithm that utilizes the estimated locations to modify the input of the state set in the reinforcement learning algorithm. Finally, we extensively evaluate our algorithms in simulation, concluding that in this imperfect vision capacity setting, our algorithms significantly improve the escape success rate of the evader.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alexopoulos, A., Schmidt, T., Badreddin, E.: Cooperative pursue in pursuit-evasion games with unmanned aerial vehicles. In: Proceedings of the IEEE international conference on intelligent robots and systems, pp. 4538–4543 (2015)

  2. Bhattacharya, S., Basar, T., Falcone, M.: Numerical approximation for a visibility based pursuit-evasion game. In: Proceedings of the IEEE international conference on intelligent robots and systems, pp. 68–75 (2014)

  3. Bhattacharya, S., Baṡar, T., Falcone, M.: Surveillance for security as a pursuit-evasion game. In: Proceedings of the international conference on decision and game theory for security, pp. 370–379 (2014)

    Google Scholar 

  4. Bopardikar, S.D., Bullo, F.: Hespanha.: On discrete-time pursuit-evasion games with sensing limitations. IEEE Trans. Robot. 24(6), 1429–1439 (2008)

    Article  Google Scholar 

  5. Chen, J., Zha, W., Peng, Z., Gu, D.: Multi-player pursuit-evasion games with one superior evader. Automatica 71, 24–32 (2016)

    Article  MathSciNet  Google Scholar 

  6. Chuong, V.N., Shahram, I., David, L.: Modeling kinect sensor noise for improved 3D reconstruction and tracking. In: IEEE international conference on 3d imaging, modeling, processing, visualization and transmission, pp. 524–530 (2012)

  7. Dukeman, A., Julie, A.A.: Hybrid mission planning with coalition formation. In: Proceedings of the international conference on autonomous agents and multiagent systems, pp. 1424–1466 (2017)

    Article  Google Scholar 

  8. Ephrati, E., Jeffrey, S.R.: Divide and conquer in multi-agent planning. In: Proceedings of the AAAI conference on artificial intelligence, pp. 385–393 (1994)

  9. Fang, B., Pan, Q., Hong, B., Ding, L., Zhong, Q., Zhang, Z.: Research on high speed evader vs. multi lower speed pursuers in multi pursuit-evasion games. Inf. Technol. J. 11(8), 989 (2012)

    Article  Google Scholar 

  10. Gerkey, B., Thrun, S., Gordon, G.: Visibility-based pursuit-evasion with limited field of view. Int. J. Robot. Res. 1, 20–27 (2010)

    Google Scholar 

  11. Gow, R.D., Renshaw, D., Findlater, K., Grant, L.A.: A comprehensive tool for modeling CMOS image-sensor-noise performance. IEEE Trans. Electron Devices 54(6), 1321–1329 (2007)

    Article  Google Scholar 

  12. Healey, G.E., Kondepudy, R.: Radiometric CCD camera calibration and noise estimation. IEEE Trans. Pattern Anal. Mach. Intell. 16(3), 267–276 (1994)

    Article  Google Scholar 

  13. Ichiro, S., Masafumi, Y.: Searching for a mobile intruder in a polygonal region. SIAM J. Comput. 21(5), 863–888 (1992)

    Article  MathSciNet  Google Scholar 

  14. Isaacs, R.: Differential games: A mathematical theory with applications to warfare and pursuit, control and optimization. Courier Corporation (1999)

  15. LaValle, S.M., Hinrichsen, J.E.: Visibility-based pursuit-evasion: The case of curved environments. IEEE Trans. Robot. Autom. 17(2), 196–202 (2001)

    Article  Google Scholar 

  16. Li, X., Peng, Z., Zha, W., Chen, J.: Construction of barrier in a three-player pursuit-evasion game. IEEE Trans. Cybern. 4(1), 1–9 (2016)

    Google Scholar 

  17. Liu, S.Y., Zhou, Z., Tomlin, C., Hedrick, J.K.: Evasion of a team of dubins vehicles from a hidden pursuer. In: Proceedings of the IEEE international conference on robotics and automation, pp. 6771–6776 (2014)

  18. Montijano, E., Sonia, M., Carlos, S.: Distributed robust data fusion based on dynamic voting. In: Proceedings of the IEEE international conference on robotics and automation, pp. 5893–5898 (2011)

  19. Murrieta-Cid, R., Muppirala, T., Sarmiento, A., Bhattacharya, S., Hutchinson, S.: Numerical approximation for a visibility based pursuit-evasion game. International Journal of Robotics Research 3, 233–253 (2007)

    Article  Google Scholar 

  20. Park, F.C., Martin, B.J.: Robot sensor calibration: solving AX= XB on the Euclidean group. IEEE Trans. Robot. Autom. 10(5), 717–721 (1994)

    Article  Google Scholar 

  21. Pfister, S.T., Kriechbaum, K.L., Roumeliotis, S.I., Burdick, J.W.: Weighted range sensor matching algorithms for mobile robot displacement estimation. In: Proceedings of the international conference on intelligent robots and systems (2002)

  22. Pierson, A., Ataei, A., Paschalidis, I.C., Schwager, M.: Cooperative multi-quadrotor pursuit of an evader in an environment with no-fly zones. In: Proceedings of the IEEE international conference on robotics and automation, pp. 320–326 (2016)

  23. Ramana, M.V., Kothari, M.: A cooperative pursuit-evasion game of a high speed evader. In: Proceedings of the IEEE conference on decision and control, pp. 2969–2974 (2015)

  24. Ramana, M.V., Kothari, M.: Pursuit-evasion games of high speed evader. J. Intell. Robot. Syst. 85(2), 293–306 (2017)

    Article  Google Scholar 

  25. René, V., Omid, S., Jin, H.K., David Hyunchul, S., Shankar, S.: Probabilistic pursuit–evasion games: Theory, implementation, and experimental evaluation. IEEE Trans Rob Autom 18(5), 662–669 (2002)

    Article  Google Scholar 

  26. Schenato, L.: Swarm coordination for pursuit evasion games using sensor networks. In: Proceedings of the IEEE international conference on robotics and automation, pp. 2493–2498 (2005)

  27. Sterling: Dutch police train eagles to snatch enemy drones. The Telegraph (2016)

  28. Stiffler, N.M., Kolling, A., O’Kane, J.M.: Persistent pursuit-evasion: The case of the preoccupied pursuer. In: Proceedings of the IEEE international conference on robotics and automation, pp. 5027–5034 (2017)

  29. Stiffler, N.M., O’Kane, J.M.: Visibility-based pursuit-evasion with probabilistic evader models. In: Proceedings of the IEEE international conference on robotics and automation, pp. 4254–4259 (2011)

  30. Stiffler, N.M., O’Kane, J.M.: A sampling-based algorithm for multi-robot visibility-based pursuit-evasion. In: Proceedings of the IEEE international conference on intelligent robots and systems, pp. 1782–1789 (2014)

  31. Stiffler, N.M., O’Kane, J.M.: Pursuit-evasion with fixed beams. In: Proceedings of the IEEE international conference on robotics and automation, pp. 4251–4258 (2016)

  32. Stiffler, N.M., O’Kane, J.M.: Complete and optimal visibility-based pursuit-evasion. Int. J. Robot. Res. 36(8), 923–946 (2017)

    Article  Google Scholar 

  33. Stroupe, A., Martin, M., Balch, T.: Distributed sensor fusion for object position estimation by multi-robot systems. In: Proceedings of the international conference on robotics and automation, pp. 1092–1098 (2001)

  34. Sun, W., Tsiotras, P.: Sequential pursuit of multiple targets under external disturbances via Zermelo–Voronoi diagrams. Automatica 81, 253–260 (2017)

    Article  MathSciNet  Google Scholar 

  35. Tan, R.: Exploiting reactive mobility for collaborative target detection in wireless sensor networks. IEEE Trans. Mob. Comput. 9(3), 317–332 (2010)

    Article  Google Scholar 

  36. Valin, J.M., Michaud, F., Rouat, J., Letourneau, D.: Robust sound source localization using a microphone array on a mobile robot. In: Proceedings of the international conference on intelligent robots and systems, pp. 1228–1233 (2003)

  37. Watkins, C., Peter, D.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)

    MATH  Google Scholar 

  38. Williams: Tokyo police are using drones with nets to catch other drones. The Telegraph (2015)

  39. Yan, F.: Pursuing a faster evader based on an agent team with unstable speeds). In: Proceedings of the international conference on autonomous agents and multiagent systems, pp. 1766–1768 (2017)

  40. Yan, F., Jiang, J., Di, K.: Multiagent pursuit-evasion problem with the pursuers moving at uncertain speeds. Journal of Intelligent & Robotic Systems, pp. 1–27 (2018)

  41. Yang, P., Ke, T., Xin, Y.: Turning high-dimensional optimization into computationally expensive optimization. IEEE Trans. Evol. Comput. 22(1), 143–156 (2018)

    Article  Google Scholar 

  42. Yao, C., Xu, M., Luo, T.: Dynamics and Control for Nonideal Solar Sails Around Artificial. J. Spacecr. Rocket. 55(3), 575–585 (2018)

    Article  Google Scholar 

  43. Yu Fan, C., Miao, L., Michael, E., Jonathan, P.H.: Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In: Proceedings of the IEEE international conference on robotics and automation, pp. 285–292 (2017)

  44. Zha, W., Chen, J., Peng, Z., Gu, D.: Construction of barrier in a fishing game with point capture. IEEE Trans. Cybern. 47(6), 1409–1422 (2017)

    Article  Google Scholar 

  45. Zhang, Y., Lynne, E.P.: IQ-ASyMTRe: Forming executable coalitions for tightly coupled multirobot tasks. IEEE Trans. Robot. 29(2), 400–416 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (61472079, 61170164, 61807008 and 61806053), the Natural Science Foundation of Jiangsu Province of China (BK20171363, BK20180356, BK20180369, BK20170693).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Southeast University, Nanjing, 211189, China

    Kai Di, Shaofu Yang, Wanyuan Wang, Fuhan Yan, Haokun Xing & Yichuan Jiang

  2. The Co-innovation Center of Shandong Universities for Future Intelligent Computing, Shandong Technology and Business University, Yantai, 264005, China

    Kai Di & Yichuan Jiang

  3. School of Computer Science and Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Jiuchuan Jiang

Authors
  1. Kai Di
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaofu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wanyuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fuhan Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haokun Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jiuchuan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yichuan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yichuan Jiang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Di, K., Yang, S., Wang, W. et al. Optimizing Evasive Strategies for an Evader with Imperfect Vision Capacity. J Intell Robot Syst 96, 419–437 (2019). https://doi.org/10.1007/s10846-019-00996-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-019-00996-1

Keywords

Search

Navigation