Abstract
Over the last few years, Man-Machine collaborative systems have been increasingly present in daily routines. In these systems, one operator usually controls the machine through explicit commands and assesses the information through a graphical user interface. Direct & implicit interaction between the machine and the user does not exist. This work presents a man-machine symbiotic concept & system where such implicit interaction is possible targeting search and rescue scenarios. Based on measuring physiological variables (e.g. body movement or electrocardiogram) through wearable devices, this system is capable of computing the psycho-physiological state of the human and autonomously identify abnormal situations (e.g. fall or stress). This information is injected into the control loop of the machine that can alter its behavior according to it, enabling an implicit man-machine communication mechanism. A proof of concept of this system was tested at the ARTEX (ARmy Technological EXperimentation) exercise organized by the Portuguese Army involving a military agent and a drone. During this event the soldier was equipped with a kit of wearables that could monitor several physiological variables and automatically detect a fall during a mission. This information was continuously sent to the drone that successfully identified this abnormal situation triggering the take-off and a situation awareness fly-by flight pattern, delivering a first-aid kit to the soldier in case he did not recover after a pre-determined time period. The results were very positive, proving the possibility and feasibility of a symbiotic system between humans and machines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Waharte, S., Trigoni, N.: Supporting search and rescue operations with UAVs. In: 2010 International Conference On Emerging Security Technologies, pp. 142–147 (2010)
Emergency Response Framework (ERF), 2nd edn. https://www.who.int/publications/i/item/9789241512299. Accessed 20 July 2023
Keane, J., Carr, S.: A brief history of early unmanned aircraft. J. Hopkins APL Tech. Dig. 32, 558–571 (2013)
Sambolek, S., Ivasic-Kos, M.: Automatic person detection in search and rescue operations using deep CNN detectors. IEEE Access. 9, 37905–37922 (2021)
Dandois, J., Olano, M., Ellis, E.: Optimal altitude, overlap, and weather conditions for computer vision UAV estimates of forest structure. Remote Sens. 7, 13895–13920 (2015)
Xu, G., Zhang, Y., Ji, S., Cheng, Y., Tian, Y.: Research on computer vision-based for UAV autonomous landing on a ship. Pattern Recogn. Lett. 30, 600-605 (2009)
Mejias, L., Correa, J., Mondragon, I., Campoy, P.: COLIBRI: a vision-guided UAV for surveillance and visual inspection. In: Proceedings 2007 IEEE International Conference On Robotics And Automation, pp. 2760–2761 (2007)
Liu, A., Mo, W., Ling, Y.: Multi-sensor environmental mapping (2016). https://patents.google.com/patent/US20160070265A1/en
Bürkle, A., Segor, F., Kollmann, M.: Towards autonomous micro UAV Swarms. J. Intell. Robot. Syst. 61, 339–353 (2011). https://doi.org/10.1007/s10846-010-9492-x
Lamont, G., Slear, J., Melendez, K.: UAV swarm mission planning and routing using multi-objective evolutionary algorithms. In: 2007 IEEE Symposium On Computational Intelligence In Multi-Criteria Decision-Making, pp. 10–20 (2007)
Doherty, P., Rudol, P.: A UAV search and rescue scenario with human body detection and geolocalization. In: AI 2007: Advances In Artificial Intelligence, pp. 1–13 (2007)
Ryan, A., Hedrick, J.: A mode-switching path planner for UAV-assisted search and rescue. In: Proceedings of the 44th IEEE Conference on Decision and Control, pp. 1471–1476 (2005)
Licklider, J.: Man-computer symbiosis. IRE Trans. Hum. Factors in Electron. HFE-1, 4–11 (1960)
SYMBIOSIS (noun) definition and synonyms | Macmillan Dictionary. https://www.macmillandictionary.com/dictionary/british/symbiosis. Accessed 20 July 2023
Han, J., Kim, J.: Human-robot interaction by reading human intention based on mirror-neuron system. In: 2010 IEEE International Conference on Robotics and Biomimetics, pp. 561–566 (2010)
Guerra, J., Dias, D., Cunha, J.P.S., COELHO, L.: Human actigraphy’s analysis through an inertial sensor. In: XXXV Congreso anual de la Sociedad Española de Ingeniería Biomédica: Libro de actas, pp. 323–326. Servicio Editorial= Argitalpen Zerbitzua (2017)
Cunha, J., Cunha, B., Pereira, A., Xavier, W., Ferreira, N., Meireles, L.: Vital-Jacket
: a wearable wireless vital signs monitor for patients’ mobility in cardiology and sports. In: 2010 4th International Conference On Pervasive Computing Technologies For Healthcare, pp. 1–2 (2010)Spilmont, E., Dias, D., Cunha, J.P.S.: AmbiUnit: a new adaptive wearable environmental monitoring device for hazardous professions. In: Inforum2018, Coimbra (2018)
Li, Q., Stankovic, J., Hanson, M., Barth, A., Lach, J., Zhou, G.: Accurate, fast fall detection using gyroscopes and accelerometer-derived posture information. In: Proceedings - 2009 6th International Workshop on Wearable and Implantable Body Sensor Networks, BSN 2009, pp. 138–143 (2009)
O que são contêineres? | Google Cloud. https://cloud.google.com/learn/what-are-containers?hl=pt-br. Accessed 12 June 2023
What Is a Container? | Docker. https://www.docker.com/resources/what-container/. Accessed 12 June 2023
Koziolek, H., Grüner, S., Rückert, J.: A comparison of MQTT brokers for distributed IoT edge computing. In: Jansen, A., Malavolta, I., Muccini, H., Ozkaya, I., Zimmermann, O. (eds.) ECSA 2020. LNCS, vol. 12292, pp. 352–368. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58923-3_23
Hassan, A.: A decentralized MQTT broker for IoT on web 3.0. Medium. (2022, 1) (2022)
Dix, P. Why Time Series Matters for Metrics, Real-Time Analytics and Sensor Data. https://get.influxdata.com/rs/972-GDU-533/images/why time series.pdf,2021,7
Grafana: The open observability platform. https://grafana.com/. Accessed 12 June 2023
About: Node-RED. https://nodered.org/about/. Accessed 12 June 2023
Ferencz, K., Domokos, J.: Rapid prototyping of IoT applications for the industry. In: 2020 IEEE International Conference on Automation, Quality and Testing, robotics (AQTR), p. 6 (2020)
Dias, A., Almeida, J., Oliveira, A., Santos, T., Martins, A., Silva, E.: Unmanned aerial vehicle for wind-turbine inspection. next step: Offshore. In: OCEANS: Hampton Roads. Hampton Roads 2022, VA, USA, pp. 1–6 (2022). https://doi.org/10.1109/OCEANS47191.2022.9977308
Silva, P., et al.: 3D Reconstruction of historical sites using an UAV. In: Proceedings of the CLAWAR (2020)
Dias, A., et al.: ROSM - Robotic Oil Spill Mitigations: OCEANS, Marseille. Marseille, France 2019, pp. 1–7 (2019). https://doi.org/10.1109/OCEANSE.2019.8867213
Azevedo, F., et al.: LiDAR-based real-time detection and modeling of power lines for unmanned aerial vehicles. Sensors 19, 1812 (2019). https://doi.org/10.3390/s19081812
: a wearable wireless vital signs monitor for patients’ mobility in cardiology and sports. In: 2010 4th International Conference On Pervasive Computing Technologies For Healthcare, pp. 1–2 (2010)Acknowledgements
This work is financed by National Funds through the Portuguese funding agency, FCT - Fundac̨ão para a Ciência e a Tecnologia, within project LA/P/0063/2020. The group also thanks the Portuguese Army and the CEMTEX group for organizing the ARTEX event and all the support during the experiment days. The authors would also like to acknowledge and thank the contribution of Prof. Bob Iannucci (Carnegie-Mellon University and Google inc.) for his seminal contributions to the early stages of the Man-Machine Symbiosis Concept.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Minhoto, V. et al. (2024). Man-Machine Symbiosis UAV Integration for Military Search and Rescue Operations. In: Marques, L., Santos, C., Lima, J.L., Tardioli, D., Ferre, M. (eds) Robot 2023: Sixth Iberian Robotics Conference. ROBOT 2023. Lecture Notes in Networks and Systems, vol 978. Springer, Cham. https://doi.org/10.1007/978-3-031-59167-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-59167-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-59166-2
Online ISBN: 978-3-031-59167-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)