Leonor Fermoselle
ABSTRACT
This paper presents a proof-of-concept of a robotic teleoperation system, that provides the human operator a thermal sense in addition to the visual sense. With a sensor suite comprising a stereo camera, 360° camera and long-wave infra-red camera, our demonstrator pushes the boundaries of virtual-reality situational awareness by bringing not only 3D visual content but also a 360° thermal experience to the operator. The visual channel of our robotic teleoperation system is represented through a head-mounted-display and the thermal channel is displayed through directional heaters in the operator cockpit and a thermal glove. Initial tests showed that an operator successfully experienced a 360° remote environment, correctly distinguished between and interacted with hot and cold objects, and could notice the presence of nearby people outside her direct field-of-view, based on their emitted heat.
- Jonathan Boisvert, Marc-Antoine Drouin, Guy Godin, and Michel Picard. 2020. Augmented reality, 3D measurement, and thermal imagery for computer-assisted manufacturing. In Emerging Digital Micromirror Device Based Systems and Applications XII, Vol. 11294. International Society for Optics and Photonics, SPIE Digital Library, United States, 112940L.Google Scholar
- Carlos Coelho, JG Tichon, Trevor J Hine, GM Wallis, and Giuseppe Riva. 2006. Media presence and inner presence: the sense of presence in virtual reality technologies. In From communication to presence: Cognition, emotions and culture towards the ultimate communicative experience. IOS Press, Amsterdam, Amsterdam, Netherlands, 25–45.Google Scholar
- I. Comsa, R. Trestian, and G. Ghinea. 2018. 360° Mulsemedia experience over next generation wireless networks – A reinforcement learning approach. In 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, United States, 1–6. https://doi.org/10.1109/QoMEX.2018.8463409Google ScholarCross Ref
- I. Comşa, E. B. Saleme, A. Covaci, G. M. Assres, R. Trestian, C. A. S. Santos, and G. Ghinea. 2020. Do I Smell Coffee? The Tale of a 360° Mulsemedia Experience. IEEE MultiMedia 27, 1 (2020), 27–36. https://doi.org/10.1109/MMUL.2019.2954405Google Scholar
- Richard L. Daft and Robert H. Lengel. 1986. Organizational information requirements, media richness and structural design. Management Science 32, 5 (1986), 554–571. https://doi.org/10.1287/mnsc.32.5.554Google ScholarDigital Library
- Richard L Daft, Robert H Lengel, and Linda Klebe Trevino. 1987. Message equivocality, media selection, and manager performance: Implications for information systems. MIS quarterly 11(1987), 355–366.Google Scholar
- Leonor Fermoselle, Simon Gunkel, Frank ter ter Haar, Sylvie Dijkstra-Soudarissanane, Alexander Toet, Omar Niamut, and Nanda van van der Stap. 2020. Let’s Get in Touch! Adding Haptics to Social VR. In ACM International Conference on Interactive Media Experiences. ACM, New York, NY, USA, 174–179.Google ScholarDigital Library
- FLIR. 2022. FLIR A655sc High-Resolution Science Grade LWIR Camera. https://www.flir.eu/products/a655sc/ Accessed: 2022-03-09.Google Scholar
- Stereolabs Inc.2022. ZED 2i – Industrial AI Stereo Camera. https://www.stereolabs.com/zed-2i/ Accessed: 2022-03-09.Google Scholar
- Seung-Won Kim, Sung Hee Kim, Choong Sun Kim, Kyoungsoo Yi, Jun-Sik Kim, Byung Jin Cho, and Youngsu Cha. 2020. Thermal display glove for interacting with virtual reality. Scientific Reports 10, 1 (2020), 1–12.Google Scholar
- Ricoh Company Ltd.2022. Ricoh Theta Z1. https://theta360.com/en/about/theta/z1.html Accessed: 2022-03-09.Google Scholar
- Honghao Lv, Geng Yang, Huiying Zhou, Xiaoyan Huang, Huayong Yang, and Zhibo Pang. 2020. Teleoperation of collaborative robot for remote dementia care in home environments. IEEE Journal of Translational Engineering in Health and Medicine 8 (2020), 1–10. https://doi.org/10.1109/JTEHM.2020.3002384Google ScholarCross Ref
- Günter Niemeyer, Carsten Preusche, Stefano Stramigioli, and Dongjun Lee. 2016. Telerobotics. Springer International Publishing, Cham, Switzerland, 1085–1108. https://doi.org/10.1007/978-3-319-32552-1_43Google Scholar
- Kaushik Parida, Hyunwoo Bark, and Pooi See Lee. 2021. Emerging thermal technology enabled augmented reality. Advanced Functional Materials 31, 39 (2021), 2007952.Google ScholarCross Ref
- Denis Peña and Fumihide Tanaka. 2020. Human perception of social robot’s emotional states via facial and thermal expressions. ACM Transactions on Human-Robot Interaction 9, 4 (2020), Article 26. https://doi.org/10.1145/3388469Google ScholarDigital Library
- Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement.Google Scholar
- Halodi Robotics. 2022. Eve Robot. https://www.halodi.com/ Accessed: 2022-03-09.Google Scholar
- Sensiks. 2022. Sensiks Sensory Reality Pods & Platform. https://www.sensiks.com Accessed: 2022-03-09.Google Scholar
- Kazuaki Tanaka, Hideyuki Nakanishi, and Hiroshi Ishiguro. 2015. Physical embodiment can produce robot operator’s pseudo presence. Frontiers in ICT 2, 8 (2015), 1–12. https://doi.org/10.3389/fict.2015.00008Google ScholarCross Ref
- Unity Technologies. 2022. Unity Real-time Development Platform. https://unity.com/ Accessed: 2022-03-09.Google Scholar
- TEGway. 2022. ThermoReal, TEGway. http://tegway.co/tegway/ Accessed: 2022-03-09.Google Scholar
- ©FLIR. 2018. Try the FLIR Virtual Reality Experience!https://www.flir.in/news-center/corporate-news/try-the-flir-virtual-reality-experience/ Accessed: 2021-08-03.Google Scholar
- Alexander Toet, Irene A. Kuling, Bouke N. Krom, and Jan B. F. van Erp. 2020. Toward enhanced teleoperation through embodiment. Frontiers in Robotics and AI 7, 14 (2020), 1–22. https://doi.org/10.3389/frobt.2020.00014Google ScholarCross Ref
- ETH Zurich. 2022. ALMA – Robotic Systems Lab, ETH Zurich. https://rsl.ethz.ch/robots-media/alma.html Accessed: 2022-03-09.Google Scholar
Recommendations
VR-Based Teleoperation for Robot Compliance Control
Robots governed by remote human operators are excellent candidates for work in hazardous or uncertain environments such as nuclear plants or outer space. For successful teleoperation, it is important to let the operator feel physically present at the ...
Virtual and Augmented Reality with Head-Tracking for Efficient Teleoperation of Groups of Robots
CW '08: Proceedings of the 2008 International Conference on CyberworldsThis paper deals with the usage of Virtual and Augmented Reality techniques in the field of teleoperation. The global context is a project for collaborative teleoperation of real robots groups. The aim is to enable to a group of teleoprators to control ...
A Real-Time Reconfigurable Collision Avoidance System for Robot Manipulation
ICMRE 2017: Proceedings of the 3rd International Conference on Mechatronics and Robotics EngineeringIntelligent robotic systems are becoming fundamental actors in industrial and hazardous facilities scenarios. Aiming to increase personnel safety and machine availability, robots can help perform repetitive and dangerous tasks which humans either prefer ...
Comments