Skip to main content
SpringerLink
Log in

Investigating Human-Robot Teams for Learning-Based Semi-autonomous Control in Urban Search and Rescue Environments

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Teams of semi-autonomous robots can provide valuable assistance in Urban Search and Rescue (USAR) by efficiently exploring cluttered environments and searching for potential victims. Their advantage over solely teleoperated robots is that they can address the task handling and situation awareness limitations of human operators by providing some level of autonomy to the multi-robot team. Our research focuses on developing learning-based semi-autonomous controllers for rescue robot teams. In this paper, we specifically investigate the influence of the operator-to-robot ratio on the performance of our proposed MAXQ hierarchical reinforcement learning based semi-autonomous controller for USAR missions. In particular, we propose a unique learning-based system architecture that allows operator control of larger numbers of rescue robots in a team as well as effective sharing of information between these robots. A rigorous comparative study of our learning-based semi-autonomous controller versus a fully teleoperation-based approach was conducted in a 3D simulation environment. The results, as expected, show that, for both semi-autonomous and teleoperation modes, the total scene exploration time increases as the number of robots utilized increases. However, when using the proposed learning-based semi-autonomous controller, the rate of exploration-time increase and operator-interaction effort are significantly lower, while task performance is significantly higher. Furthermore, an additional case study showed that our learning-based approach can provide more scene coverage during robot exploration when compared to a non-learning based method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Casper, J., Micire, M., Murphy, R.: Issues in intelligent robots for search and rescue. In: Proceedings SPIE Unmanned Ground Vehicle Technology II, pp 292–302 (2000)

  2. Casper, J., Murphy, R.: Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Trans. Syst. Man Cybern. B 33(3), 367–385 (2003)

    Article  Google Scholar 

  3. Liu, Y., Nejat, G.: Robotic urban search and rescue: a survey from the control perspective. J. Intell. Robot. Syst. 72(2), 147–165 (2013)

    Article  Google Scholar 

  4. Wong, C., Seet, G., Sim, S.: Multiple-robot systems for USAR: key design attributes and deployment issues. Int. J. Adv. Robot. Syst. 8(1), 85–101 (2011)

    Article  Google Scholar 

  5. Gatsoulis, Y., Virk, G.S., Dehghani-Sanij, A.A.: On the measurement of situation awareness for effective human-robot interaction in teleoperated systems. J. Cogn. Eng. Decis. Mak. 4(1), 69–98 (2010)

    Article  Google Scholar 

  6. Adams, J.A.: Multiple robot/single human interaction: effects on perceived workload. Behav. Inf. Technol. 28 (2), 183–198 (2009)

    Article  Google Scholar 

  7. de Visser, E., Parasuraman, R.: Adaptive aiding of human-robot teaming effects of imperfect automation on performance, trust, and workload. J. Cogn. Eng. Decis. Mak. 5(2), 209–231 (2011)

    Article  Google Scholar 

  8. Chen, J.Y.C., Durlach, P.J., Sloan, J.A., Bowens, L.D.: Human-robot interaction in the context of simulated route reconnaissance missions. Mil. Psychol. 20(3), 135–149 (2008)

    Article  Google Scholar 

  9. Breslow, L.A., Gartenberg, D., McCurry, J.M., Trafton, J.G.: Dy7namic operator overload: a model for predicting workload during supervisory control. IEEE Trans. Hum. Mach. Syst. 44(1), 30–40 (2014)

    Article  Google Scholar 

  10. McKendrick, R., Shaw, T, de Visser, E., Sager, H., Kidwell, B., Parasuraman, R.: Team performance in networked supervisory control of unmanned air vehicles effects of automation, working memory, and communication content. Hum. Factors 56(3), 463–475 (2014)

    Article  Google Scholar 

  11. Fincannon, T.D., Evans, A.W., Phillips, E., Jentsch, F., Keebler, J.: The influence of team size and communication modality on team effectiveness with unmanned systems. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 419–423 (2009)

  12. de Visser, E., Kidwell, B., Payne, J., Lu, L., Parker, J., Brooks, N., Chabuk, T., Spriggs, S., Freedy, A., Scerri, P., Parasuraman, R.: Best of both worlds design and evaluation of an adaptive delegation interface. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 255–259 (2013)

  13. Cummings, M.L., Bertucelli, L.F., Macbeth, J., Surana, A.: Task versus vehicle-based control paradigms in multiple unmanned vehicle supervision by a single operator. IEEE Trans. Hum. Mach. Syst. 44(3), 353–361 (2014)

    Article  Google Scholar 

  14. Prewett, M.S., Saboe, K.N., Johnson, R.C., Coovert, M.D., Elliot, L.R.: Workload in human-robot interaction: a review of manipulations and outcomes. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 1393–1397 (2009)

  15. Velagapudi, P., Scerri, P., Sycara, K., Wang, H., Lewis, M., Wang, J.: Scaling effects in multi-robot control. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2121–2126 (2008)

  16. Hsieh, M.A., Cowley, A., Keller, J.F., Chaimowicz, L., Grocholsky, B., Kumar, V., Taylor, C.J., Endo, Y., Arkin, R.C., Jung, B., Wolf, D.F., Sukhatme, G.S., MacKenzie, D.C.: Adaptive teams of autonomous aerial and ground robots for situation awareness. J. Field Robot. 24(11), 991–1014 (2007)

    Article  Google Scholar 

  17. Zhang, K., Xiaobo, L.: Human-robot team coordination that considers human fatigue. J. Adv. Robot. Syst. 11(6) (2014)

  18. Olsen, D.R., Goodrich, M.A.: Metrics for evaluating human–robot interactions. In: Proceedings of the PERMIS (2003)

  19. Gatsoulis, Y., Virk, G., Dehghani-Sanij, A.: On the measurement of situation awareness for effective human-robot interaction in teleoperated systems. J. Cogn. Eng. Dec. Mak. 4(1), 69–98 (2010)

    Article  Google Scholar 

  20. Gao, F., Cummings, M.L., Solovey, E.T.: Modeling teamwork in supervisory control of multiple robots. IEEE Trans. Hum. Mach. Syst. 44(4), 441–453 (2014)

    Article  Google Scholar 

  21. Lewis, M., Wang, H., Chien, S.Y.: Process and performance in human-robot teams. J. Cogn. Eng. Decis. Mak. 5(2), 186–208 (2011)

    Article  Google Scholar 

  22. Gunn, T., Anderson, J.: Dynamic heterogeneous team formation for robotic urban search and rescue. J. Comp. Syst. Sci. 81(3), 553–567 (2015)

    Article  Google Scholar 

  23. Rosenfeld, A., Agmon, N., Maksimov, O., Azaria, A., Kraus, S.: Intelligent agent supporting human-multi-robot team collaboration. In: Proceedings of the AAMAS Workshop ARMS (2015)

  24. Nourjou, R., Smith, S.F., Hatayama, M., Szekely, P.: Intelligent algorithm for assignment of agents to human strategy in centralized multi-agent coordination. J. Softw. 9(10), 2586–2597 (2014)

    Article  Google Scholar 

  25. Chen, J.Y.C., Barnes, M.: Supervisory control of multiple robots: effects of imperfect automation and individual differences. Hum. Factors: J. Hum. Factors Ergon. Soc. 54(2), 157–174 (2012)

    Article  Google Scholar 

  26. Vilela, J., Liu, Y., Nejat, G.: Semi-autonomous exploration with robot teams in urban search and rescue. In: IEEE International Symposium on Safety, Security, and Rescue Robotics, pp 1–6 (2013)

  27. Doroodgar, B., Liu, Y., Nejat, G.: A learning-based semi-autonomous controller for robotic exploration of unknown disaster scenes while searching for victims. IEEE Trans. Cybern. 44(12), 2719–2732 (2014)

    Article  Google Scholar 

  28. Niroui, F., Sprenger, B., Nejat, G.: Robot exploration in unknown cluttered environments when dealing with uncertainty. In: Proceedings of the IEEE International Symposium on Robotics and Intelligent Sensors (2017)

  29. Liu, Y., Nejat, G., Doroodgar, B.: Learning-based semi-autonomous control for robots in urban search and rescue. In: Proceedings of the IEEE International Symposium on Safety, Security, and Rescue Robotics, pp 1–6 (2012)

  30. Liu, Y., Nejat, G.: Multirobot cooperative learning for semiautonomous control in urban search and rescue applications. J. Field Robot. 33(4), 512–536 (2016)

    Article  Google Scholar 

  31. Brooks, N., Scerri, P., Sycara, K.: Asynchronous control with ATR for large robot teams. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 444–448 (2011)

  32. Wang, H., Lewis, M., Chien, S., Velagapudi, P.: Scaling effects for synchronous vs. asynchronous video in multi-robot search. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 364–368 (2009)

  33. Sato, N., Matsuno, F., Yamasaki, T., Kamegawa, T., Shiroma, N., Igarashi, H.: Cooperative task execution by a multiple robot team and its operators in search and rescue operations. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 1083–1088 (2004)

  34. Lewis, M., Wang, H., Chien, S.Y.: Choosing autonomy modes for multirobot search. Hum. Factors 52 (2), 225–233 (2010)

    Article  Google Scholar 

  35. Mouaddib, A., Zilberstein, S., Beynier, A., Jeanpierre, L.: A decision-theoretic approach to cooperative control and adjustable autonomy. In: Proceedings of the European Conference on Artificial Intelligence, pp 971–972 (2010)

  36. Côté, N., Canu, A., Bouzid, M., Mouaddib, A.: Humans-robots sliding collaboration control in complex environments with adjustable autonomy. In: Proceedings of the IEEE/WIC/ACM International Conference Web Intelligence and Intelligent Agent Technology, pp 146–153 (2012)

  37. Wray, K.H., Pineda, L., Zilberstein, S.: Hierarchical approach to transfer of control in semi-autonomous systems. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp 517–523 (2016)

  38. Zhang, K., Collins, E.G., Shi, D.: Centralized and distributed task allocation in multi-robot teams via a stochastic clustering auction. J. Auton. Adapt. Syst. 7(2) (2012)

  39. Zhang, K., Collins, E.G., Barbu, A.: An efficient stochastic clustering auction for heterogenous robotic collaborative teams. J. Intell. Robot. Syst. 72(3–4), 541–558 (2013)

    Article  Google Scholar 

  40. Zhang, K., Collins, E.G., Barbu, A.: A novel stochastic clustering auction for task allocation in multi-robot teams. In: Proceedings of the IEEE/RSJ International Conference on Intelligent and Robotic Systems, pp 3300–3307 (2010)

  41. Zhang, K., Collins, E.G., Shi, D., Liu, X., Chuy, O.: A stochastic clustering auction (SCA) for centralized and distributed task allocation in multi-agent teams. Distrib. Auton. Robot. Syst. 8, 345–354 (2009)

    Google Scholar 

  42. Zhang, Z., Littman, M., Chen, X.: Covering number as a complexity measure for POMDP planning and learning. In: AAAI Conference on Artificial Intelligence, pp 1853–1859 (2012)

  43. Dietterich, D.: Hierarchical reinforcement learning with MAXQ value function decomposition. J. Artif. Intell. Res. 13, 227–303 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  44. Liu, Y., Nejat, G., Vilela, J.: Learning to cooperate together: a semi-autonomous control architecture for multi-robot teams in urban search and rescue. In: IEEE International Symposium on Safety, Security, and Rescue Robotics (2013)

  45. Doroodgar, B., Nejat, G.: A hierarchical reinforcement learning based control architecture for semi-autonomous rescue robots in cluttered environments. In: IEEE International Conference on Automation Science and Engineering, pp 948–953 (2010)

  46. Chien, S., Wang, H., Lewis, M.: Effects of alarms on control of robot teams. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 434–438 (2011)

  47. Oppermann, R.: Adaptive User Support: Ergonomic Design of Manually and Automatically Adaptable Software. CRC Press, Boca Raton (1994)

    Google Scholar 

  48. Parasuraman, R., Galster, S., Squire, P., Furukawa, H., Miller, C.: A flexible delegation-type interface enhances system performance in human supervision of multiple robots: empirical studies with RoboFlag. IEEE Trans. Cybern. 35(4), 481–493 (2005)

    Article  Google Scholar 

  49. Clare, A., Cummings, M., How, J., Whitten, A., Toupet, O.: Operator object function guidance for a real-time unmanned vehicle scheduling algorithm. J. Aerosp. Comput. Inf. Commun. 9(4), 161–173 (2012)

    Article  Google Scholar 

  50. Sourceforge. USARSim. [Online]. http://sourceforge.net/projects/usarsim/

  51. Epic Games Inc. UDK. [Online]. http://www.unrealengine.com/udk

  52. Lewis, M., Wang, J., Hughes, S.: USARSim: simulation for the study of human-robot interaction. J. Cognitive Eng. Decis. Mak. 1(1), 98–120 (2007)

    Article  Google Scholar 

  53. Wang, J., Lewis, M., Hughes, S., Koes, M.: Validating USARsim for use in HRI Research. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 457–461 (2005)

  54. Carpin, S., Lewis, M., Wang, J., Balakirsky, S., Scrapper, C.: USARSim: a robot simulator for research and education. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 1400–1405 (2007)

  55. Balaguer, B., Balakirsky, S., Carpin, S., Visser, A.: Evaluating maps produced by urban search and rescue robots: lessons learned from RoboCup. J. Auton. Robot. 27(4), 449–464 (2009)

    Article  Google Scholar 

  56. Shiroma, N., Chiu, Y., Sato, N., Matsuno, F.: Cooperative task execution of a search and rescue mission by a multi-robot team. J. Adv. Robot. 19(3), 311–329 (2005)

    Article  Google Scholar 

  57. Calisi, D., Farinelli, A., Iocchi, L., Nardi, D.: Multi-objective exploration and search for autonomous rescue robots. J. Field Robot. 24(8-9), 763–777 (2007)

    Article  Google Scholar 

  58. Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., Goodrich, M.: Common metrics for human-robot interaction. In: Proceeding of the ACM SIGCHI/SIGART Conference on Human-robot Interaction, pp 33–40 (2006)

  59. Donmez, B., Pina, P.E., Cummings, M.L.: Evaluation criteria for human-automation performance metrics. In: Performance Evaluation and Benchmarking of Intelligent Systems, ch. 2, pp 21–40 (2009)

  60. Tubre, T.C., Collins, J.M.: Jackson and Schuler (1985) Revisited: a meta-analysis of the relationships between role ambiguity, role conflict, and job performance. J. Manag. 26(1), 155–169 (2000)

    Google Scholar 

  61. Barde, M.P., Barde, P.J.: What to use to express the variability of data: Standard deviation or standard error of mean Perspect. Clin. Res. 3(3), 113–116 (2012)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This work was funded by the Natural Science and Engineering Research Council of Canada (NSERC) and the Canada Research Chairs (CRC) Program.

Author information

Authors and Affiliations

  1. Autonomous Systems and Biomechatronics Laboratory, Department of Mechanical and Industrial Engineering, University of Toronto, 5 King’s College Rd, Toronto, Ontario, M5S 3G8, Canada

    A. Hong, O. Igharoro, Y. Liu, F. Niroui & G. Nejat

  2. Department of Mechanical and Industrial Engineering, University of Toronto, 5 King’s College Rd, Toronto, Ontario, M5S 3G8, Canada

    B. Benhabib

Authors
  1. A. Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. O. Igharoro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Y. Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. F. Niroui
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. G. Nejat
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. B. Benhabib
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Niroui.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hong, A., Igharoro, O., Liu, Y. et al. Investigating Human-Robot Teams for Learning-Based Semi-autonomous Control in Urban Search and Rescue Environments. J Intell Robot Syst 94, 669–686 (2019). https://doi.org/10.1007/s10846-018-0899-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-018-0899-0

Keywords

Search

Navigation