Skip to main content
SpringerLink
Log in

A Heuristic Learning Algorithm for Preferential Area Surveillance by Unmanned Aerial Vehicles

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

A heuristic learning algorithm is presented in this paper to solve the problem of persistent preferential surveillance by an Unmanned Aerial Vehicle (UAV). The algorithm helps a UAV perform surveillance as per quantitative priority specifications over a known area. It allows the specification of regional priorities either as percentages of visitation to be made by a UAV to each region or as percentages of surveillance time to be spent within each. Additionally, the algorithm increases the likelihood of target detection in an unknown area. The neighborhood of a detected target is suspected to be a region of a high likelihood of target detection, and the UAV plans its path accordingly to verify this suspicion. Similar to using the target information, the algorithm uses the risk information to reduce the frequency of visits to risky regions. The technique of using risk map to avoid risky regions is adapted from the existing geometric reinforcement learning technique. The effectiveness of this algorithm is demonstrated using simulation results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Nigam, N., Bieniawski, S., Kroo, I., Vian, J.: Control of multiple UAVs for persistent surveillance: Algorithm and flight test results. IEEE Trans. Control Syst. Technol. 20(5), 1236–1251 (2012)

    Article  Google Scholar 

  2. Mersheeva, V., Friedrich, G.: Multi-UAV monitoring with priorities and limited energy resources, Jerusalem, Israel (2015)

  3. Bethke, B., How, J.P., Vian, J.: Multi-UAV persistent surveillance with communication constraints and health management. In: AIAA Guidance, Navigation, and Control Conference, Chicago, Illinois (2009)

  4. Stump, E., Michael, N.: Multi-robot persistent surveillance planning as a vehicle routing problem. In: Proceedings of IEEE Conference on Automation Science and Engineering (CASE), pp. 569–575, Trieste (2011)

  5. Alamdari, S., Fata, E., Smith, S.L.: Persistent monitoring in discrete environments: Minimizing the maximum weighted latency between observations. Int. J. Robot. Res. 33(1), 138–154 (2014)

    Article  Google Scholar 

  6. Cassandras, C.G., Lin, X.: Optimal control of multi-agent persistent monitoring systems with performance constraints. In: Proceedings of the 50th EEE Conference on Decision and Control, pp. 2907–2912 (2011)

  7. Arvelo, E., Kim, E., Martins, N.C.: Memoryless Control Design for Persistent Surveillance under Safety Constraints. Cornell University Library Article. arXiv: 1209.5805. Accessed 20 May 2016

  8. Leahy, K., Zhou, D., Vasile, C.-I., Oikonomopoulos, K., Schwager, M., Belta, C.: Provably correct persistent surveillance for unmanned aerial vehicles subject to charging constraints. In: Proceedings of the International Symposium on Experimental Robotics (ISER 14), Marrakech, Morocco (2014)

  9. Burman, J., Hespanha, J., Madhow, U., Isaacs, J., Venkateswaran, S., Pham, T.: Autonomous UAV persistent surveillance using bio-inspired strategies. In: Proceedings of the SPIE 8389, Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR III 838917 (2012)

  10. Nigam, N.: The multiple unmanned air vehicle persistent surveillance problem: a review. Machines 20(1), 13–72 (2014)

    Article  Google Scholar 

  11. Russell, S.J., Norvig, P.: Learning from Examples. In: Russell, S.J., Norvig, P. (eds.) Artificial Intelligence: A Modern Approach. 3rd edn, pp. 695–696. Pearson Education Inc (2010)

  12. Sutton, R.S., Barto, A.G.: Introduction. In: Sutton, R.S., Barto, A.G. (eds.) Reinforcement Learning: An Introduction, pp. 3–9. The MIT Press, Cambridge (1998)

  13. Zhang, B., Mao, Z., Liu, W., Liu, J.: Geometric reinforcement learning for path planning of UAV. J. Intell. Robot. Syst. 77, 391–409 (2015)

    Article  Google Scholar 

  14. Geramifard, A., Redding, J., How, J.P.: Intelligent cooperative control architecture: a framework for performance improvement using safe learning. J. Intell. Robot. Syst. 72, 83–103 (2013)

    Article  Google Scholar 

  15. Cook, K., Bryan, E., Yu, H., Bai, H., Seppi, K., Beard, R.: Intelligent cooperative control for urban tracking. J. Intell. Robot. Syst. 74, 251–267 (2014)

    Article  Google Scholar 

  16. Hirsch, M.J., Ortiz-Pena, H.J., Eck, C.: Cooperative tracking of multiple targets by a team of autonomous UAVs. doi:10.4018/joris.2012010104 (2012)

  17. Ure, N.K., Chowdhary, G., Chen, Y.F., Cutler, M., How, J.P., Vian, J.: Decentralized learning-based planning for multi-agent missions in the presence of actuator failures. In: International Conference on Unmanned Aircraft Systems, GA Atlanta (2013)

  18. Polycarpou, M.M., Yang, Y., Passino, K.M.: Multi-UAV cooperative search using an opportunistic learning method. J. Dyn. Syst. Meas. Contr. 129(5), 716–728 (2007)

    Article  Google Scholar 

  19. Yu, H., Beard, R., Argyle, M., Chamberlain, C.: Probabilistic path planning for Cooperative Target Tracking using aerial and ground vehicles. In: American Control Conference, San Francisco CA (2011)

  20. Sujit, P.B., Ghose, D.: Self Assessment-Based decision making for multiagent cooperative search. IEEE Trans. Autom. Sci. Eng. 8(4), 705–719 (2011)

    Article  Google Scholar 

  21. Zhu, J.J., Xu, X.: Biopsychically inspired cognitive control for autonomous agents based on motivated learning. In: 17th International Conference on Methods and Models in Automation and Robotics, Miedzyzdrojie (2012)

  22. Polycarpou, M.M., Yang, Y., Passino, K.M: Cooperative Control of Distributed Multi-agent Systems. IEEE Control Syst. Mag. (2001)

  23. Redding, J., Bethke, B., Bertuccelli, L.F., How, J.P.: Active learning in persistent surveillance uav missions. In: AIAA Infotech@Aerospace Conference, Seattle, Washington (2009)

  24. Ramasamy, M., Ghose, D.: Learning-based preferential surveillance algorithm for persistent surveillance by unmanned aerial vehicles. In: 2016 International Conference on Unmanned Aircraft Systems (ICUAS), Arlington, VA (2016)

Download references

Author information

Authors and Affiliations

  1. Department of Aerospace Engineering, Indian Institute of Science, Bangalore, 560012, Karnataka, India

    Debasish Ghose

  2. Aeronautical Development Establishment, Defence Research and Development Organization, Bangalore, 560093, Karnataka, India

    Manickam Ramasamy

Authors
  1. Manickam Ramasamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debasish Ghose
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Debasish Ghose.

Additional information

This paper is an extension of the work presented by the authors in [24].

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramasamy, M., Ghose, D. A Heuristic Learning Algorithm for Preferential Area Surveillance by Unmanned Aerial Vehicles. J Intell Robot Syst 88, 655–681 (2017). https://doi.org/10.1007/s10846-017-0498-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-017-0498-5

Keywords

Mathematics Subject Classification 2010

Search

Navigation