1 Introduction

The automotive human-machine interface (HMI) has gradually become one of the critical issues in the automotive design field [1]. Tan and Zhao [2] studied the changes in human-automobile interaction modalities with respect to the automobile interior space, human-machine interface, and processes of operation and interaction. Other researchers [3, 4] focused on the design objects and modalities in automotive HMI, and concluded that the traditional priorities of industrial design, with interior layout and in-vehicle information systems (IVIS) as separated parts, are giving way to a trend of increasing integration, with car styling designers, UI designers, and third party developers (e.g., makers) working more closely and becoming increasingly involved in the design process. Against this backdrop, aiding design teams by using integrated tools for innovative conceptual design has taken on great importance [5]. On the one hand, design tools enable designers to quickly respond to user needs, thus reducing the costs of data collection. On the other hand, with the change from a modality relying on a single individual expert to those centering on self-organizing, which facilitates nimble and flexible adaptation to changing design environments, these tools are foreseen to further enable communication and sharing of concepts among designers [6]. Therefore, this research begins with the storyboard tool, which is widely used in automotive HMI design as the starting point from which to develop a small-scale design tool that can be used across users of automotive HMI and user knowledge.

Design is essentially a process of collecting, storing, and applying knowledge [7]. Viewed from the theoretical standpoint of the semantics of information encoding and decoding, knowledge related to design can be classified into two categories: design knowledge and user knowledge [8]. In 2001, Sato and Sakol argued that the design process is a knowledge implementation process through which a designer engages in design behavior to transform internal knowledge into external product attributes and physical manifestations [9]. Sakol further advanced the idea by arguing that the design process is a continuous cycle of user knowledge becoming design knowledge and vice versa [10]. Furthermore, when designers apply internal knowledge during the problem-solving process, the knowledge itself takes on the functions of tools [11].

Automotive HMI design knowledge includes car styling knowledge, human-computer interface knowledge, automotive human factors engineering, knowledge of the psychology of driving, and other descriptive knowledge related to the design domain [12]. User knowledge is the experience of solving problems, and is gained when using the product(s); it is linked with the user’s needs, behaviors, and mental model. User knowledge pertaining to automotive HMI design is a kind of procedural knowledge of how users accomplish driving tasks, which is intimately related to driver behavior and operational procedures, and organized as a task flow through the arrangement of key tasks in a logical, time sequenced manner. The vehicle information systems that are presently being introduced have led to the functional enhancement of the automotive HMI such that in addition to driving, users are also required to engage in numerous other activities unrelated to driving such as interacting with the navigation, social communication, and entertainment systems. This has increased the complexity of driving [13]. Therefore, the acquisition and description of procedural knowledge related to users’ driving is the key factor aiding the conceptual design of automotive HMI. This knowledge has the attributes of methods, plans, and resources; design tools derived from and inspired by user knowledge aids designers in understanding users’ driving behavior and their interaction context. This approach improves the design efficiency of inter-group collaboration between car styling designers and UI designers. Figure 1 shows the knowledge related to automotive HMI design.

Fig. 1.
figure 1

Knowledge relevant to automotive HMI

Full size image

2 Storyboard Tool

Storyboards were widely used in the early stage of movie filming as a tool for assisting in screenplay and scene visualization. Storyboards use conceptual diagramming to visualize individual shots thus allowing the entire production team to visualize camera angles, scene and location changes, spatial arrangements, lighting, movement, and other elements before actual shooting. Sherman pointed out that the creation of movie storyboards involves diagramming a series of rough sketches and providing an explanation for every scene and each position within scenes. Storyboards are a visual record of the entire film prior to shooting. Hart considered the storyboard to be an effective tool that offers a way to set up every shot and scene during filming [14]. Metz refers to camera shots, which is the basic element of the storyboard, as “the basic unit of communication through film.” Typically, the organization of shots and scenes is based on a logical narrative sequence with the storyboard explaining each action of each scene. Visualization by means of storyboard expression permits the production team members to improve their understanding of complex movements and positions in a scene. Therefore, the storyboard is also known as the “visual script” [15].

As a very expressive creative tool, storyboards have also been used in the field of design. (1) The storyboards gives rise to improved communication between different roles. During the design process, the designers’ concepts can be transmitted via storyboards to the entire design team, clients, and potential users. This frame-by-frame method assists designers in describing issues and communicating solutions. The use of accessible, visual language enables people of different backgrounds to comprehend the creative concepts. (2) Similar to a rough sketch, the storyboard creating process is also a visual thought process. Garner asserted that the form and function of storyboards would change as the product development process evolves [16]. (3) From the standpoint of finding solutions to design demands, storyboards reflect the divergence in design thinking, that is, the evolution of the solutions and deduction process of designers. The function of the storyboard evolves with each stage of creative design of the product; it is an effective way of activating conceptual ideas throughout the design process.

In the automotive HMI conceptual design process, storyboards can “sequentialize” the observable features of user knowledge with a series of driver behaviors and operations, thus forming a process flow that creates a comprehensive design solution that integrates system functions, interactive process flow, and interface design.

3 Method

3.1 Storyboard Framework Based on the Axes of Vision, Scenario, and Process

In movie and animation production, the storyboard, with a subtitle script, is usually divided into four axes: picture, action, dialog, and time. Each axis separately describes different aspects of a character, a scene, time, or other elements, thereby fully presenting each kind of information [17]. This paper draws on the multi-dimensional axes model used in movie storyboard making to gain user knowledge, by using axes of vision, scenario, and process, to build a storyboard for the automotive HMI. This storyboard is expected to help realize the goals of rapidly comprehending user behavior and transforming them into design solution concepts. The framework of the storyboard presented in this paper is shown in Fig. 2.

Fig. 2.
figure 2

Storyboard framework

Full size image

The vision axis describes user knowledge from the perspective of user groups and their needs; it consists of characters, events, interfaces, and other elements. The visual axis features the visual communication and event-based narrative flow of traditional storyboards, all of which help designers to understand designated users and design tasks.

The scenario axis describes the procedural knowledge of users in the operational state of interactive tasks and is composed of the environment, behavioral, and other elements. It is the procedural and knowledge transformation of people and events in the visual axis. It offers conceptual assistance to designers in dealing with interactive behavior and interactive semantics.

The process axis describes the sequence of interactive tasks from the operational point of view and is composed of operational processes. It is the transformation of procedural knowledge into the design, and the transformation of auxiliary interface information, interaction flow, interaction mode, and usage cases into the design.

The axes of vision, scenario, and process constitute a new interactive interface design storyboard for the automotive HMI; it is based on the traditional visual narrative storyboard to describe and communicate the procedural knowledge of drivers, thereby sequentially and logically promoting the transformation of the interactive process, interface styling, and visualization into design.

3.2 Interactive Design of Automotive HMI Based on Storyboard

According to the Theory of Design Processes proposed by Tovey in 2005 [18], storyboard-based design activities can be abstracted into two categories: analytics-user knowledge acquisition, and synthesis-design program generation.

  1. 1.

    User knowledge acquisition: Storyboards function as intermediary tools to promote the transformation of user knowledge into design. The storyboard transforms user knowledge into three types of useful design information: in the vision axis – the needs and intentions of the user (driver), in the scenario axis - the status of the user’s solving and executing in the task, and in the procedural axis – the information on the operational time of the task. The layered storyboard framework permits extraction of implicit user knowledge for explicit representation to aid auxiliary HMI styling and to the benefit of UI designers as they seek to obtain and incorporate the user’s driving-related procedural knowledge.

  2. 2.

    The creation of a design solution is the result of knowledge processing to drive design, which here mainly refers to the integrated design of display, control, and information systems related to automotive HMI. Figure 3 shows the storyboard-based design solving process of user knowledge acquisition and proposal generation.

    Fig. 3.
    figure 3

    Design solving process based on storyboard construction

    Full size image

4 Case Study of Navigation Interface UI Design

In this work, the case analysis method was used to verify the theoretical framework of the above research. The studied case is based on the UI design of a navigation interface for an independently developed electric vehicle. Navigation is an important function of the interactive system of electric vehicles. The key issues in this design are: (1) the functional framework of navigation and (2) the conceptual UI design of the navigation interface.

4.1 Constructing an Interactive Navigation Storyboard Based on User Knowledge

  1. 1.

    Building the vision axis: Based on field research in the car and its environment, this work analyzed the behavior in relation to driving tasks and interaction with the navigation system, and extracted procedural knowledge related to the operations from it, and then formulated it as a storyboard to create a script, which is displayed in seven frames. The vision axis shows the entire process from the moment the user starts until they finish a navigation task. This visualization of procedural knowledge helps designers rapidly understand user needs and intentions, and paves the way for cognitive transformation on the scenario axis. Figure 4 shows the vision axis.

    Fig. 4.
    figure 4

    Storyboard for UI design of the navigation interface in an electric car

    Full size image
  2. 2.

    Building the scenario axis: Based on the vision axis, and as shown on the scenario axis of Fig. 4, the overall procedure is: start the car - input destination - … - end navigation. The message conveyed by the vision axis shows that the interactive tasks of navigation route planning are dynamically associated with the driving environment. For example, prompts such as “turn right 200 m before the upcoming fork in the road” uses changes in the road environment to trigger the user’s anticipation of the task. The extraction of task objectives from the scenario axis of the task helps designers understand the characteristics of real-time interactive tasks according to the dynamic driving conditions, as to promote the subsequent knowledge transformation of the means of interaction and procedures. Figure 4 shows the scenario axis.

  3. 3.

    Building the process axis: Based on the set of the vision axis, the basic process is: initiate navigation - turn on the head-up display (HUD) - voice input of destination - load destination - route preview - route confirmation - end navigation. The details are as shown in the process axis in Fig. 4. Depending on real-time traffic conditions, there may be incremental interactive tasks in the navigation process, such as switching views and repeating voice announcements. The designer’s understanding and design optimization of the operational process in these tasks can enhance the user’s emotional experience. Therefore, the process axis not only establishes the chronological sequence of the interactive tasks but also coordinates the interaction between the car and environment through the scenario axis, thus helping designers to improve their understanding of the dynamic navigation decision-making process and supporting the display and control interface design in later stages.

4.2 Concept UI Design of the Navigation Interface with the Use of Storyboards

The functional framework and the concept design of interactions are as follows:

  1. 1.

    The functional framework design is shown in Fig. 5. Based on the user knowledge transformation in the storyboard tool axes of vision, scenario, and process, it can be observed that there are different levels of interactive tasks. Basic tasks correspond to route planning functions (destination input, route selection, preview, etc.); incremental tasks correspond to decision-supporting functions (view switching, zooming, etc.). The next step involves designing the form of interaction corresponding to the functional framework and tasks.

    Fig. 5.
    figure 5

    Navigation functional framework and form of interaction

    Full size image
  2. 2.

    The conceptual UI design of navigation is shown in Fig. 4. In the early research and development stage, designers working on the interactive aspects would have developed a general set of gestures and information frameworks for the information systems functions of the electric vehicle such as those for music playback and answering the phone. In this context, navigation decision-supporting functions that are controlled imprecisely (i.e., view switching, volume control, and so forth), and route planning functions (destination input) will be operated using a combination of interaction through a touch-screen in the center console and the voice channel. The general design cue is to reduce the cognitive load during the driving process via the multi-channel integration of voice commands and gestures and create a functional framework for navigation with more natural man-machine interactions. In line with the functional framework, interface styling designers work on the number of function keys and the interface layout, as shown in Fig. 6, based on the concept sketches of the navigation control interface on the steering wheel.

    Fig. 6.
    figure 6

    Concept sketches of multi-functional steering wheel navigation control interface

    Full size image

Navigation HMI design: With the help of the storyboard showing the interactions with the navigation system, the designers subsequently designed the display and control interface. The hardware part of the navigation interface includes the shapes, layout, color, and materials. In accordance with the principle of functional importance, the designers positioned the navigation activation key on the upper right side of the central spoke of the steering wheel to facilitate operation with the push of a thumb by the driver. According to the principle of functional similarity, the navigation eye-level display button is positioned above the navigation start button, and the button is designed to feature a textured matte material, which is distinct from the glossy material of the navigation keys. This approach would enable the driver to find the position on the control interface according to different functional needs without having to lower their head and looking away from the road. With regard to the interactive gesture interface of the navigation system, the UI designers integrated the functional framework, interaction procedure, and gesture set (shown in Fig. 7) and utilized experienced user testing and design iteration. The interface design of the heads-up display (HUD) in the navigation system includes information architecture and visual design. In terms of its content, the navigational information is organized according to the principle of cognitive economy. To reduce interference, in accordance with visual design strategy, the sizes, proportions, and colors of the arrow symbols, numbers, and characters in the visual interface were repeatedly scrutinized. Figure 8 shows the design rendering of the control interface integrated navigation HUD with the multi-function steering wheel.

Fig. 7.
figure 7

Interactive gesture design for navigational functions (team solutions)

Full size image
Fig. 8.
figure 8

Rendering of the interactive navigation interface

Full size image

5 Discussion

This study used the case analysis method for the interactive storyboard design of electric vehicles to investigate the use of the storyboard tool and method in designing automotive HMI based on user knowledge. The research shows that the storyboard can play the role of an intermediate tool of conceptual thinking integration during the process of solution development. This research proposed a storyboard framework based on the axes of vision, scenario, and process. The vision axis is a visual description of the user’s intention and behavior. The scenario axis describes the user’s procedural knowledge pertaining to the execution of the interactive task. The process axis describes the sequence of interactive tasks from the operational point of view and is the concrete manifestation of the procedural knowledge present in the scenario axis. The storyboard tool proposed in this research is based on the traditional narrative storyboard used in visual media; the tool facilitates the use of procedural, temporal, and logical driver knowledge in the transformation into the design of interactive processes, interface styling, and visuals.

6 Conclusion

The development of automotive HMI is headed towards software and hardware integration and intelligent systems. In an agile, open, and collaborative design environment, the exploration of design tools capable of assisting cross-domain teams to seek out user needs and create solutions quickly is worth studying. This paper presented a new form of the storyboard tool based on the incorporation of user knowledge with design solution transformation in the solution-seeking process, thus driving innovations in HMI, interactive processes, and the form of interaction. The paper also presented a case study of the design of an interactive navigation interface in an electric automobile. The results were verified by applying the tool and method in the design process; thus, it can be concluded that this research enabled the development of automotive HMI design tools with a vital theoretical and practical foundation.