Hyeonggeun Yun
ABSTRACT
This paper presents design guidelines for teleoperated driving interfaces within computational driver assistance systems for unstructured environments. The purpose of the guidelines is to manage the unpredictability of computational model-based assistance in unstructured environments in order to reduce user workload. Thus, we conducted a user study to evaluate workload and obtain insights into both the advantages and disadvantages of the computational driver assistance system in order to develop the guidelines. The study utilized a deep learning-based driver assistance method in simulated environments to observe the workload of users while teleoperated driving with the assistance method. Based on the user study, we proposed guidelines for teleoperated driving interface with computational driver assistance systems. We anticipate that the proposed guidelines could improve the understanding of computational driver assistance systems and reduce the workload of teleoperated driving in unstructured environments, thereby enhancing driver’s trust as well as comfort.
- S Anderson, S Peters, Karl Iagnemma, and James Overholt. 2010. Semi-autonomous stability control and hazard avoidance for manned and unmanned ground vehicles. In Proceedings of the 27th Army Science Conference. 1–8.Google Scholar
- Ola Benderius, Christian Berger, and Victor Malmsten Lundgren. 2017. The best rated human–machine interface design for autonomous vehicles in the 2016 grand cooperative driving challenge. IEEE Transactions on intelligent transportation systems 19, 4 (2017), 1302–1307. https://doi.org/10.1109/TITS.2017.2749970Google ScholarCross Ref
- Yujia Cao, Angela Mahr, Sandro Castronovo, Mariët Theune, Christoph Stahl, and Christian A. Müller. 2010. Local Danger Warnings for Drivers: The Effect of Modality and Level of Assistance on Driver Reaction. In Proceedings of the 15th International Conference on Intelligent User Interfaces (Hong Kong, China) (IUI ’10). Association for Computing Machinery, New York, NY, USA, 239–248. https://doi.org/10.1145/1719970.1720004Google ScholarDigital Library
- Jessie YC Chen, Ellen C Haas, and Michael J Barnes. 2007. Human performance issues and user interface design for teleoperated robots. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37, 6 (2007), 1231–1245. https://doi.org/10.1109/TSMCC.2007.905819Google ScholarDigital Library
- Younggeol Cho, Hyeonggeun Yun, Jinwon Lee, Arim Ha, and Jihyeok Yun. 2023. GoonDAE: Denoising-Based Driver Assistance for Off-Road Teleoperation. IEEE Robotics and Automation Letters 8, 4 (2023), 2405–2412. https://doi.org/10.1109/LRA.2023.3250008Google ScholarCross Ref
- Tok-Son Choe, Jin-Bae Park, Sang-Hyun Joo, and Yong-Woon Park. 2014. 1D virtual force field algorithm for reflexive local path planning of mobile robots. Electronics Letters 50, 20 (2014), 1429–1430. https://doi.org/10.1049/el.2014.2787Google ScholarCross Ref
- Mark Colley, Svenja Krauss, Mirjam Lanzer, and Enrico Rukzio. 2021. How Should Automated Vehicles Communicate Critical Situations? A Comparative Analysis of Visualization Concepts. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 3, Article 94 (sep 2021), 23 pages. https://doi.org/10.1145/3478111Google ScholarDigital Library
- Na Du, Feng Zhou, Dawn Tilbury, Lionel Peter Robert, and X. Jessie Yang. 2021. Designing Alert Systems in Takeover Transitions: The Effects of Display Information and Modality. In 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Leeds, United Kingdom) (AutomotiveUI ’21). Association for Computing Machinery, New York, NY, USA, 173–180. https://doi.org/10.1145/3409118.3475155Google ScholarDigital Library
- Terrence Fong, Charles Thorpe, and Charles Baur. 2001. Advanced interfaces for vehicle teleoperation: Collaborative control, sensor fusion displays, and remote driving tools. Autonomous Robots 11 (2001), 77–85. https://doi.org/10.1023/A:1011212313630Google ScholarDigital Library
- Jean-Michael Georg and Frank Diermeyer. 2019. An adaptable and immersive real time interface for resolving system limitations of automated vehicles with teleoperation. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE, 2659–2664. https://doi.org/10.1109/SMC.2019.8914306Google ScholarDigital Library
- Jean-Michael Georg, Johannes Feiler, Frank Diermeyer, and Markus Lienkamp. 2018. Teleoperated Driving, a Key Technology for Automated Driving? Comparison of Actual Test Drives with a Head Mounted Display and Conventional Monitors. In 2018 21st International Conference on Intelligent Transportation Systems (ITSC). IEEE, 3403–3408. https://doi.org/10.1109/ITSC.2018.8569408Google ScholarDigital Library
- Gaetano Graf, Yomna Abdelrahman, Hao Xu, Yasmeen Abdrabou, Dmitrij Schitz, Heinrich Hußmann, and Florian Alt. 2020. The Predictive Corridor: A Virtual Augmented Driving Assistance System for Teleoperated Autonomous Vehicles. In International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE). 61–69.Google Scholar
- Gaetano Graf and Heinrich Hussmann. 2020. User Requirements for Remote Teleoperation-Based Interfaces. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Virtual Event, DC, USA) (AutomotiveUI ’20). Association for Computing Machinery, New York, NY, USA, 85–88. https://doi.org/10.1145/3409251.3411730Google ScholarDigital Library
- Gaetano Graf, Henri Palleis, and Heinrich Hussmann. 2020. A Design Space for Advanced Visual Interfaces for Teleoperated Autonomous Vehicles. In Proceedings of the International Conference on Advanced Visual Interfaces (Salerno, Italy) (AVI ’20). Association for Computing Machinery, New York, NY, USA, Article 71, 3 pages. https://doi.org/10.1145/3399715.3399942Google ScholarDigital Library
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9Google ScholarCross Ref
- Tove Helldin, Göran Falkman, Maria Riveiro, and Staffan Davidsson. 2013. Presenting System Uncertainty in Automotive UIs for Supporting Trust Calibration in Autonomous Driving. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Eindhoven, Netherlands) (AutomotiveUI ’13). Association for Computing Machinery, New York, NY, USA, 210–217. https://doi.org/10.1145/2516540.2516554Google ScholarDigital Library
- Kai Holländer, Philipp Wintersberger, and Andreas Butz. 2019. Overtrust in External Cues of Automated Vehicles: An Experimental Investigation. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Utrecht, Netherlands) (AutomotiveUI ’19). Association for Computing Machinery, New York, NY, USA, 211–221. https://doi.org/10.1145/3342197.3344528Google ScholarDigital Library
- Tim Horberry, Christine Mulvihill, Michael Fitzharris, Brendan Lawrence, Mike Lenné, Jonny Kuo, and Darren Wood. 2022. Human-Centered Design for an In-Vehicle Truck Driver Fatigue and Distraction Warning System. IEEE Transactions on Intelligent Transportation Systems 23, 6 (2022), 5350–5359. https://doi.org/10.1109/TITS.2021.3053096Google ScholarCross Ref
- Amin Hosseini and Markus Lienkamp. 2016. Enhancing telepresence during the teleoperation of road vehicles using HMD-based mixed reality. In 2016 IEEE Intelligent vehicles symposium (IV). IEEE, 1366–1373. https://doi.org/10.1109/IVS.2016.7535568Google ScholarDigital Library
- Amin Hosseini, Florian Richthammer, and Markus Lienkamp. 2016. Predictive Haptic Feedback for Safe Lateral Control of Teleoperated Road Vehicles in Urban Areas. In 2016 IEEE 83rd Vehicular Technology Conference (VTC Spring). IEEE, 1–7. https://doi.org/10.1109/VTCSpring.2016.7504430Google ScholarCross Ref
- Jinglu Jiang, Alexander J Karran, Constantinos K Coursaris, Pierre-Majorique Léger, and Joerg Beringer. 2023. A situation awareness perspective on human-AI interaction: Tensions and opportunities. International Journal of Human–Computer Interaction 39, 9 (2023), 1789–1806. https://doi.org/10.1080/10447318.2022.2093863Google ScholarCross Ref
- Chunhui Jing, Chenguang Shang, Dongyu Yu, Yaodong Chen, and Jinyi Zhi. 2022. The impact of different AR-HUD virtual warning interfaces on the takeover performance and visual characteristics of autonomous vehicles. Traffic injury prevention 23, 5 (2022), 277–282. https://doi.org/10.1080/15389588.2023.2224910Google ScholarCross Ref
- Nihan Karatas, Takahiro Tanaka, Kazuhiro Fujikakc, Yuki Yoshihara, Hitoshi Kanamori, Yoshitaka Fuwamoto, and Morihiko Yoshida. 2020. Evaluation of AR-HUD interface during an automated intervention in manual driving. In 2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2158–2164. https://doi.org/10.1109/IV47402.2020.9304610Google ScholarCross Ref
- Siddartha Khastgir, Stewart Birrell, Gunwant Dhadyalla, and Paul Jennings. 2019. Effect of knowledge of automation capability on trust and workload in an automated vehicle: a driving simulator study. In Advances in Human Aspects of Transportation: Proceedings of the AHFE 2018 International Conference on Human Factors in Transportation, July 21-25, 2018, Loews Sapphire Falls Resort at Universal Studios, Orlando, Florida, USA 9. Springer International Publishing, 410–420. https://doi.org/10.1007/978-3-319-93885-1_37Google ScholarCross Ref
- Sunjun Kim, Byungjoo Lee, Thomas van Gemert, and Antti Oulasvirta. 2020. Optimal Sensor Position for a Computer Mouse. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376735Google ScholarDigital Library
- Suresh Kolekar, Shilpa Gite, Biswajeet Pradhan, and Abdullah Alamri. 2022. Explainable AI in scene understanding for autonomous vehicles in unstructured traffic environments on Indian roads using the inception U-Net Model with Grad-CAM visualization. Sensors 22, 24 (2022), 9677. https://doi.org/10.3390/s22249677Google ScholarCross Ref
- Jieun Lee, Genya Abe, Kenji Sato, and Makoto Itoh. 2021. Developing human-machine trust: Impacts of prior instruction and automation failure on driver trust in partially automated vehicles. Transportation research part F: traffic psychology and behaviour 81 (2021), 384–395. https://doi.org/10.1016/j.trf.2021.06.013Google ScholarCross Ref
- Domagoj Majstorović, Simon Hoffmann, Florian Pfab, Andreas Schimpe, Maria-Magdalena Wolf, and Frank Diermeyer. 2022. Survey on Teleoperation Concepts for Automated Vehicles. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 1290–1296. https://doi.org/10.1109/SMC53654.2022.9945267Google ScholarCross Ref
- Philipp Mayring. 2019. Qualitative content analysis: Demarcation, varieties, developments. Forum: Qualitative Social Research 20, 3 (2019), 1–26. https://doi.org/10.17169/fqs-20.3.3343Google ScholarCross Ref
- Dave Miller, Annabel Sun, and Wendy Ju. 2014. Situation awareness with different levels of automation. In 2014 IEEE International Conference on systems, man, and cybernetics (SMC). IEEE, 688–693. https://doi.org/10.1109/SMC.2014.6973989Google ScholarCross Ref
- Dylan Moore, Rebecca Currano, G. Ella Strack, and David Sirkin. 2019. The Case for Implicit External Human-Machine Interfaces for Autonomous Vehicles. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Utrecht, Netherlands) (AutomotiveUI ’19). Association for Computing Machinery, New York, NY, USA, 295–307. https://doi.org/10.1145/3342197.3345320Google ScholarDigital Library
- John Morrell and Kamil Wasilewski. 2010. Design and evaluation of a vibrotactile seat to improve spatial awareness while driving. In 2010 IEEE Haptics Symposium. IEEE, 281–288. https://doi.org/10.1109/HAPTIC.2010.5444642Google ScholarDigital Library
- Claudia Trinidad Moscoso Paredes, Trond Foss, and Gunnar Jenssen. 2021. Phantom braking in Advanced Driver Assistance Systems. Driver experience and Car manufacturer warnings in Owner manuals. Technical Report. SINTEF.Google Scholar
- Giovanni Mosiello, Andrey Kiselev, and Amy Loutfi. 2013. Using augmented reality to improve usability of the user interface for driving a telepresence robot. Paladyn, Journal of Behavioral Robotics 4, 3 (2013), 174–181. https://doi.org/doi:10.2478/pjbr-2013-0018Google ScholarCross Ref
- Abhishek Mukhopadhyay, Vinay Krishna Sharma, Prashant Tatyarao Gaikwad, Ajay Kumar Sandula, and Pradipta Biswas. 2022. Exploring the Use of XR Interfaces for Driver Assistance in Take Over Request. In Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Seoul, Republic of Korea) (AutomotiveUI ’22). Association for Computing Machinery, New York, NY, USA, 58–61. https://doi.org/10.1145/3544999.3552527Google ScholarDigital Library
- Michael A. Nees, Nithya Sharma, and Karli Herwig. 2020. Some Characteristics of Mental Models of Advanced Driver Assistance Systems: A Semi-structured Interviews Approach. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 64, 1 (2020), 1313–1317. https://doi.org/10.1177/1071181320641314Google ScholarCross Ref
- Stefan Neumeier, Philipp Wintersberger, Anna-Katharina Frison, Armin Becher, Christian Facchi, and Andreas Riener. 2019. Teleoperation: The Holy Grail to Solve Problems of Automated Driving? Sure, but Latency Matters. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Utrecht, Netherlands) (AutomotiveUI ’19). Association for Computing Machinery, New York, NY, USA, 186–197. https://doi.org/10.1145/3342197.3344534Google ScholarDigital Library
- Luis Oliveira, Jacob Luton, Sumeet Iyer, Chris Burns, Alexandros Mouzakitis, Paul Jennings, and Stewart Birrell. 2018. Evaluating How Interfaces Influence the User Interaction with Fully Autonomous Vehicles. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Toronto, ON, Canada) (AutomotiveUI ’18). Association for Computing Machinery, New York, NY, USA, 320–331. https://doi.org/10.1145/3239060.3239065Google ScholarDigital Library
- Raja Parasuraman. 2000. Designing automation for human use: empirical studies and quantitative models. Ergonomics 43, 7 (2000), 931–951. https://doi.org/10.1080/001401300409125Google ScholarCross Ref
- Keunwoo Park, Eunhye Youn, Sunbum Kim, Yeonsu Kim, and Geehyuk Lee. 2021. Design of acceleration feedback of UGV using a Stewart platform. In 2021 21st International Conference on Control, Automation and Systems (ICCAS). IEEE, 79–82. https://doi.org/10.23919/ICCAS52745.2021.9648847Google ScholarDigital Library
- Sami Park, Yilun Xing, Kumar Akash, Teruhisa Misu, and Linda Ng Boyle. 2022. The Impact of Environmental Complexity on Drivers’ Situation Awareness. In Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Seoul, Republic of Korea) (AutomotiveUI ’22). Association for Computing Machinery, New York, NY, USA, 131–138. https://doi.org/10.1145/3543174.3546831Google ScholarDigital Library
- Daniel J Rea and James E Young. 2019. Backseat teleoperator: affective feedback with on-screen agents to influence teleoperation. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 19–28. https://doi.org/10.1109/HRI.2019.8673014Google ScholarCross Ref
- Smit Saparia, Andreas Schimpe, and Laura Ferranti. 2021. Active safety system for semi-autonomous teleoperated vehicles. In 2021 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops). IEEE, 141–147. https://doi.org/10.1109/IVWorkshops54471.2021.9669239Google ScholarCross Ref
- Andreas Schimpe, Domagoj Majstorovic, and Frank Diermeyer. 2022. Steering Action-aware Adaptive Cruise Control for Teleoperated Driving. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 988–993. https://doi.org/10.1109/SMC53654.2022.9945081Google ScholarCross Ref
- Dmitrij Schitz, Shuai Bao, Dominik Rieth, and Harald Aschemann. 2021. Shared autonomy for teleoperated driving: A real-time interactive path planning approach. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 999–1004. https://doi.org/10.1109/ICRA48506.2021.9561918Google ScholarDigital Library
- Dmitrij Schitz, Gaetano Graf, Dominik Rieth, and Harald Aschemann. 2021. Model-Predictive Cruise Control for Direct Teleoperated Driving Tasks. In 2021 European Control Conference (ECC). IEEE, 1808–1813. https://doi.org/10.23919/ECC54610.2021.9655187Google ScholarCross Ref
- Kohei Sonoda and Takahiro Wada. 2017. Displaying system situation awareness increases driver trust in automated driving. IEEE Transactions on Intelligent Vehicles 2, 3 (2017), 185–193. https://doi.org/10.1109/TIV.2017.2749178Google ScholarCross Ref
- Jork Stapel, Alexandre Gentner, and Riender Happee. 2022. On-road trust and perceived risk in Level 2 automation. Transportation research part F: traffic psychology and behaviour 89 (2022), 355–370. https://doi.org/10.1016/j.trf.2022.07.008Google ScholarCross Ref
- Jork Stapel, Freddy Antony Mullakkal-Babu, and Riender Happee. 2019. Automated driving reduces perceived workload, but monitoring causes higher cognitive load than manual driving. Transportation research part F: traffic psychology and behaviour 60 (2019), 590–605. https://doi.org/10.1016/j.trf.2018.11.006Google ScholarCross Ref
- Nicholas Stevens, Paul M Salmon, Guy H Walker, and Neville A Stanton. 2016. Off the beaten track: situation awareness in experienced and novice off-road drivers. In Proceedings of the 2016 Australasian Road Safety Conference. Australasian College of Road Safety, Australia, 1–3.Google Scholar
- Justin Storms. 2017. Modeling and improving teleoperation performance of semi-autonomous wheeled robots. Ph. D. Dissertation. University of Michigan, Ann Arbor, MI, USA.Google Scholar
- Krzysztof Adam Szczurek, Raul Marin Prades, Eloise Matheson, Jose Rodriguez-Nogueira, and Mario Di Castro. 2023. Multimodal Multi-User Mixed Reality Human–Robot Interface for Remote Operations in Hazardous Environments. IEEE Access 11 (2023), 17305–17333. https://doi.org/10.1109/ACCESS.2023.3245833Google ScholarCross Ref
- António Tavares, Jose Luis Silva, and Rodrigo Ventura. 2023. Physiologically Attentive User Interface for Improved Robot Teleoperation. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23). Association for Computing Machinery, New York, NY, USA, 776–789. https://doi.org/10.1145/3581641.3584084Google ScholarDigital Library
- Felix Tener and Joel Lanir. 2022. Driving from a Distance: Challenges and Guidelines for Autonomous Vehicle Teleoperation Interfaces. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 250, 13 pages. https://doi.org/10.1145/3491102.3501827Google ScholarDigital Library
- Yohai Trabelsi, Or Shabat, Joel Lanir, Oleg Maksimov, and Sarit Kraus. 2023. Advice Provision in Teleoperation of Autonomous Vehicles. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23). Association for Computing Machinery, New York, NY, USA, 750–761. https://doi.org/10.1145/3581641.3584068Google ScholarDigital Library
- Nicole Trübswetter and Klaus Bengler. 2013. Why should I use ADAS? Advanced driver assistance systems and the elderly: knowledge, experience and usage barriers. In Driving Assesment Conference, Vol. 7. 495–501.Google ScholarCross Ref
- Warren J von Eschenbach. 2021. Transparency and the black box problem: Why we do not trust AI. Philosophy & Technology 34, 4 (2021), 1607–1622. https://doi.org/10.1007/s13347-021-00477-0Google ScholarCross Ref
- Marcel Walch, Kristin Lange, Martin Baumann, and Michael Weber. 2015. Autonomous Driving: Investigating the Feasibility of Car-Driver Handover Assistance. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Nottingham, United Kingdom) (AutomotiveUI ’15). Association for Computing Machinery, New York, NY, USA, 11–18. https://doi.org/10.1145/2799250.2799268Google ScholarDigital Library
- Gesa Wiegand, Malin Eiband, Maximilian Haubelt, and Heinrich Hussmann. 2020. “I’d like an Explanation for That!”Exploring Reactions to Unexpected Autonomous Driving. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (Oldenburg, Germany) (MobileHCI ’20). Association for Computing Machinery, New York, NY, USA, Article 36, 11 pages. https://doi.org/10.1145/3379503.3403554Google ScholarDigital Library
- Sen Zhu, Guangming Xiong, Huiyan Chen, and Jianwei Gong. 2021. Guidance Point Generation-Based Cooperative UGV Teleoperation in Unstructured Environment. Sensors 21, 7 (2021), 2323. https://doi.org/10.3390/s21072323Google ScholarCross Ref
Index Terms
- Driving with Black Box Assistance: Teleoperated Driving Interface Design Guidelines for Computational Driver Assistance Systems in Unstructured Environments
Recommendations
Effects of smartphone based advanced driver assistance system on distracted driving behavior
Nowadays smartphone distractions are frequently encountered while driving and are causing an increasing number of crashes. A method to prevent them is related to the use of advanced driver assistance systems (ADAS) which ensures that the driver's ...
Comments