skip to main content
10.1145/3640792.3675705acmconferencesArticle/Chapter ViewAbstractPublication PagesautomotiveuiConference Proceedingsconference-collections
research-article

Introducing AV-Sketch: An Immersive Participatory Design Tool for Automated Vehicle — Passenger Interaction

Published: 22 September 2024 Publication History

Abstract

In the emerging automated vehicle (AV)—passenger interaction domain, there is no agreed-upon set of methods to design early concepts. Non-designers may find it challenging to brainstorm interfaces for unfamiliar technology like AVs. Therefore, we explore using an immersive virtual environment to enable expert and non-expert designers to actively participate in the design phases. We built AV-Sketch, an in-situ (on-site) simulator that allows the creation of automotive interfaces while being immersed in VR depicting diverse AV-passenger interactions. At first, we conducted a participatory design study (N=15) by utilizing PICTIVE (Plastic Interface for Collaborative Technology) to conceptualize human-machine interfaces for AV passengers. The findings led to the design of AV-Sketch, which we tested in a design session (N=10), assessing users’ design experiences. Overall, participants felt more engaged and confident with the in-situ experience, enabling better contextualization of design ideas in real-world scenarios, with improved spatial considerations and dynamic aspects of in-vehicle interfaces.

References

[1]
Rima Sabina Aouf. 2016. Google spins off self-driving car project into separate company Waymo. Retrieved 14.04.2024 from https://www.dezeen.com/2016/12/14/google-spins-off-self-driving-car-project-separate-company-waymo/
[2]
Ashratuz Zavin Asha, Fahim Anzum, Patrick Finn, Ehud Sharlin, and Mario Costa Sousa. 2020. Designing External Automotive Displays: VR Prototypes and Analysis. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Virtual Event, DC, USA) (AutomotiveUI ’20). Association for Computing Machinery, New York, NY, USA, 74–82. https://doi.org/10.1145/3409120.3410658
[3]
Ashratuz Zavin Asha and Ehud Sharlin. 2023. Designing Inclusive Interaction with Autonomous Vehicles for Older Passengers. In Proceedings of the 2023 ACM Designing Interactive Systems Conference (Pittsburgh, PA, USA) (DIS ’23). Association for Computing Machinery, New York, NY, USA, 2138–2154. https://doi.org/10.1145/3563657.3596045
[4]
Ashratuz Zavin Asha, Christopher Smith, Georgina Freeman, Sean Crump, Sowmya Somanath, Lora Oehlberg, and Ehud Sharlin. 2021. Co-Designing Interactions between Pedestrians in Wheelchairs and Autonomous Vehicles. In Designing Interactive Systems Conference 2021 (Virtual Event, USA) (DIS ’21). Association for Computing Machinery, New York, NY, USA, 339–351. https://doi.org/10.1145/3461778.3462068
[5]
Ashratuz Zavin Asha, Christopher Smith, Lora Oehlberg, Sowmya Somanath, and Ehud Sharlin. 2020. Views from the Wheelchair: Understanding Interaction between Autonomous Vehicle and Pedestrians with Reduced Mobility. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3383041
[6]
Jelte E Bos, Scott N MacKinnon, and Anthony Patterson. 2005. Motion sickness symptoms in a ship motion simulator: effects of inside, outside, and no view. Aviation, space, and environmental medicine 76, 12 (2005), 1111–1118.
[7]
John Brooke 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4–7.
[8]
Erin Cherry and Celine Latulipe. 2014. Quantifying the Creativity Support of Digital Tools through the Creativity Support Index. ACM Trans. Comput.-Hum. Interact. 21, 4, Article 21 (jun 2014), 25 pages. https://doi.org/10.1145/2617588
[9]
Mark Colley, Christian Bräuner, Mirjam Lanzer, Marcel Walch, Martin Baumann, and Enrico Rukzio. 2020. Effect of Visualization of Pedestrian Intention Recognition on Trust and Cognitive Load. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Virtual Event, DC, USA) (AutomotiveUI ’20). ACM, Association for Computing Machinery, New York, NY, USA, 181–191. https://doi.org/10.1145/3409120.3410648
[10]
Mark Colley, Benjamin Eder, Jan Ole Rixen, and Enrico Rukzio. 2021. Effects of Semantic Segmentation Visualization on Trust, Situation Awareness, and Cognitive Load in Highly Automated Vehicles. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, Article 155, 11 pages. https://doi.org/10.1145/3411764.3445351
[11]
Mark Colley, Pascal Jansen, Enrico Rukzio, and Jan Gugenheimer. 2022. SwiVR-Car-Seat: Exploring Vehicle Motion Effects on Interaction Quality in Virtual Reality Automated Driving Using a Motorized Swivel Seat. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 4, Article 150 (dec 2022), 26 pages. https://doi.org/10.1145/3494968
[12]
Mark Colley, Svenja Krauss, Mirjam Lanzer, and Enrico Rukzio. 2021. How Should Automated Vehicles Communicate Critical Situations? A Comparative Analysis of Visualization Concepts. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 3, Article 94 (sep 2021), 23 pages. https://doi.org/10.1145/3478111
[13]
Mark Colley, Max Rädler, Jonas Glimmann, and Enrico Rukzio. 2022. Effects of Scene Detection, Scene Prediction, and Maneuver Planning Visualizations on Trust, Situation Awareness, and Cognitive Load in Highly Automated Vehicles. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 2, Article 49 (jul 2022), 21 pages. https://doi.org/10.1145/3534609
[14]
Mark Colley, Marcel Walch, and Enrico Rukzio. 2019. For a Better (Simulated) World: Considerations for VR in External Communication Research. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings (Utrecht, Netherlands) (AutomotiveUI ’19). Association for Computing Machinery, New York, NY, USA, 442–449. https://doi.org/10.1145/3349263.3351523
[15]
Juliet Corbin and Anselm Strauss. 2015. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (4th ed.). SAGE Publications, Inc, Chapter 12, 220–236. Open Coding: Identifying Concepts.
[16]
Henriette Cornet, Sebastian Stadler, Penny Kong, Goran Marinkovic, Fritz Frenkler, and Peer Sathikh. 2019. User-centred design of autonomous mobility for public transportation in Singapore. Transportation Research Procedia (2019). https://api.semanticscholar.org/CorpusID:213601918
[17]
Tomás Dorta, Gokce Kinayoglu, and Michael Hoffmann. 2016. Hyve-3D and the 3D Cursor: Architectural co-design with freedom in Virtual Reality. International Journal of Architectural Computing 14, 2 (2016), 87–102. https://doi.org/10.1177/1478077116638921
[18]
Tomás Dorta, Stéphane Safin, Sana Boudhraâ, and Emmanuel Beaudry Marchand. 2019. Co-Designing in Social VR. Process awareness and suitable representations to empower user participation. ArXiv abs/1906.11004 (2019). https://api.semanticscholar.org/CorpusID:180261493
[19]
Tobias Drey, Jan Gugenheimer, Julian Karlbauer, Maximilian Milo, and Enrico Rukzio. 2020. VRSketchIn: Exploring the Design Space of Pen and Tablet Interaction for 3D Sketching in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376628
[20]
Ana Maria Bustamante Duarte, Nina Brendel, Auriol Degbelo, and Christian Kray. 2018. Participatory Design and Participatory Research: An HCI Case Study with Young Forced Migrants. ACM Trans. Comput.-Hum. Interact. 25, 1, Article 3 (feb 2018), 39 pages. https://doi.org/10.1145/3145472
[21]
Aaron Gluck, Kwajo Boateng, Earl W. Huff Jr., and Julian Brinkley. 2020. Putting Older Adults in the Driver Seat: Using User Enactment to Explore the Design of a Shared Autonomous Vehicle. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Virtual Event, DC, USA) (AutomotiveUI ’20). Association for Computing Machinery, New York, NY, USA, 291–300. https://doi.org/10.1145/3409120.3410645
[22]
David Goedicke, Alexandra W.D. Bremers, Sam Lee, Fanjun Bu, Hiroshi Yasuda, and Wendy Ju. 2022. XR-OOM: MiXed Reality Driving Simulation with Real Cars for Research and Design. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 107, 13 pages. https://doi.org/10.1145/3491102.3517704
[23]
Amr Gomaa, Guillermo Reyes, Alexandra Alles, Lydia Rupp, and Michael Feld. 2020. Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle. In Proceedings of the 2020 International Conference on Multimodal Interaction (Virtual Event, Netherlands) (ICMI ’20). Association for Computing Machinery, New York, NY, USA, 501–509. https://doi.org/10.1145/3382507.3418817
[24]
Kim Halskov and Nicolai Brodersen Hansen. 2015. The diversity of participatory design research practice at PDC 2002–2012. International Journal of Human-Computer Studies 74 (2015), 81–92. https://doi.org/10.1016/j.ijhcs.2014.09.003
[25]
Renate Häuslschmid, Max von Bülow, Bastian Pfleging, and Andreas Butz. 2017. Supporting Trust in Autonomous Driving. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (Limassol, Cyprus) (IUI ’17). Association for Computing Machinery, New York, NY, USA, 319–329. https://doi.org/10.1145/3025171.3025198
[26]
Philipp Hock, Mark Colley, Ali Askari, Tobias Wagner, Martin Baumann, and Enrico Rukzio. 2022. Introducing VAMPIRE - Using Kinaesthetic Feedback in Virtual Reality for Automated Driving Experiments. In 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications(AutomotiveUI ’22). Association for Computing Machinery, New York, NY, USA, 9 pages. https://doi.org/10.1145/3543174.3545252
[27]
Ming Hou, Karthik Mahadevan, Sowmya Somanath, Ehud Sharlin, and Lora Oehlberg. 2020. Autonomous Vehicle-Cyclist Interaction: Peril and Promise. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376884
[28]
Riitta Jääskeläinen. 2010. Think-aloud protocol. Handbook of translation studies 1 (2010), 371–374.
[29]
Jie Li, Vinoba Vinayagamoorthy, Raz Schwartz, Wijnand IJsselsteijn, David A. Shamma, and Pablo Cesar. 2020. Social VR: A New Medium for Remote Communication and Collaboration. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3375160
[30]
Arminda Guerra Lopes. 2016. Using research methods in human computer interaction to design technology for resilience. Journal of Information Systems and Technology Management 13 (2016), 363–388. https://api.semanticscholar.org/CorpusID:32888768
[31]
Karthik Mahadevan, Elaheh Sanoubari, Sowmya Somanath, James E. Young, and Ehud Sharlin. 2019. AV-Pedestrian Interaction Design Using a Pedestrian Mixed Traffic Simulator. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS ’19). Association for Computing Machinery, New York, NY, USA, 475–486. https://doi.org/10.1145/3322276.3322328
[32]
Karthik Mahadevan, Sowmya Somanath, and Ehud Sharlin. 2018. Communicating Awareness and Intent in Autonomous Vehicle-Pedestrian Interaction. Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174003
[33]
Piergiuseppe Mallozzi, Patrizio Pelliccione, Alessia Knauss, Christian Berger, and Nassar Mohammadiha. 2019. Autonomous Vehicles: State of the Art, Future Trends, and Challenges. Springer International Publishing, Cham, 347–367. https://doi.org/10.1007/978-3-030-12157-0_16
[34]
Mark McGill, Graham Wilson, Daniel Medeiros, and Stephen Anthony Brewster. 2022. PassengXR: A Low Cost Platform for Any-Car, Multi-User, Motion-Based Passenger XR Experiences. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 2, 15 pages. https://doi.org/10.1145/3526113.3545657
[35]
Yanni Mei, Jie Li, Huib de Ridder, and Pablo Cesar. 2021. CakeVR: A Social Virtual Reality (VR) Tool for Co-designing Cakes. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 572, 14 pages. https://doi.org/10.1145/3411764.3445503
[36]
Michael J. Muller. 1991. PICTIVE—an Exploration in Participatory Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New Orleans, Louisiana, USA) (CHI ’91). Association for Computing Machinery, New York, NY, USA, 225–231. https://doi.org/10.1145/108844.108896
[37]
Luis Oliveira, Jacob Luton, Sumeet Iyer, Chris Burns, Alexandros Mouzakitis, Paul Jennings, and Stewart Birrell. 2018. Evaluating How Interfaces Influence the User Interaction with Fully Autonomous Vehicles. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Toronto, ON, Canada) (AutomotiveUI ’18). Association for Computing Machinery, New York, NY, USA, 320–331. https://doi.org/10.1145/3239060.3239065
[38]
Andreas Riegler, Andreas Riener, and Clemens Holzmann. 2021. Augmented Reality for Future Mobility: Insights from a Literature Review and HCI Workshop. i-com 20, 3 (2021), 295–318. https://doi.org/
[39]
Peter A. M. Ruijten, Jacques M. B. Terken, and Sanjeev N. Chandramouli. 2018. Enhancing Trust in Autonomous Vehicles through Intelligent User Interfaces That Mimic Human Behavior. Multimodal Technologies and Interaction (2018). https://doi.org/10.3390/mti2040062
[40]
Kristin E. Schaefer and Edward R. Straub. 2016. Will passengers trust driverless vehicles? Removing the steering wheel and pedals. In 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA). IEEE, New York, NY, USA, 159–165. https://doi.org/10.1109/COGSIMA.2016.7497804
[41]
Marc Aurel Schnabel and Shuva Chowdhury. 2020. VR Unmatched - Leveraging Non-experts as Co-Urban Designers. CAADRIA proceedings (2020). https://api.semanticscholar.org/CorpusID:222130550
[42]
Brandon Schoettle and Michael Sivak. 2014. A survey of public opinion about autonomous and self-driving vehicles in the US, the UK, and Australia. Technical Report. University of Michigan, Ann Arbor, Transportation Research Institute. https://www.vtpi.org/avip.pdf
[43]
J.P. Thalen and M.C. van der Voort. 2012. Facilitating User Involvement in Product Design Through Virtual Reality. In Virtual Reality, Xin‐Xing Tang (Ed.). IntechOpen, Rijeka, Chapter 6. https://doi.org/10.5772/48602
[44]
Jean Toner. 2009. Small is not too Small: Reflections Concerning the Validity of Very Small Focus Groups (VSFGs). Qualitative Social Work 8, 2 (June 2009), 179–192. https://doi.org/10.1177/1473325009103374
[45]
Philipp Wintersberger, Anna-Katharina Frison, Andreas Riener, and Tamara von Sawitzky. 2019. Fostering User Acceptance and Trust in Fully Automated Vehicles: Evaluating the Potential of Augmented Reality. PRESENCE: Virtual and Augmented Reality 27, 1 (2019), 46–62. https://doi.org/10.1162/pres_a_00320 arXiv:https://doi.org/10.1162/pres_a_00320
[46]
Marcel Woide, Mark Colley, Nicole Damm, and Martin Baumann. 2022. Effect of System Capability Verification on Conflict, Trust, and Behavior in Automated Vehicles. In Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Seoul, Republic of Korea) (AutomotiveUI ’22). Association for Computing Machinery, New York, NY, USA, 119–130. https://doi.org/10.1145/3543174.3545253
[47]
Dohyeon Yeo, Gwangbin Kim, and Seungjun Kim. 2020. Toward Immersive Self-Driving Simulations: Reports from a User Study across Six Platforms. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376787

Index Terms

  1. Introducing AV-Sketch: An Immersive Participatory Design Tool for Automated Vehicle — Passenger Interaction

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    AutomotiveUI '24: Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
    September 2024
    438 pages
    ISBN:9798400705106
    DOI:10.1145/3640792
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 22 September 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    Author Tags

    1. Automated vehicle
    2. VR design tool
    3. participatory design

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • Natural Sciences and Engineering Research Council of Canada (NSERC)

    Conference

    AutomotiveUI '24
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 248 of 566 submissions, 44%

    Upcoming Conference

    AutomotiveUI '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 106
      Total Downloads
    • Downloads (Last 12 months)106
    • Downloads (Last 6 weeks)19
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media