Keywords

1 Introduction

Multi-screen vehicle system is the development trend of automatic driving and semi-automatic driving. With the popularization of L1 and L2 autonomous driving technologies, the driving system gradually assumes the driver’s driving tasks [1]. At the same time, the vehicle central system is no longer isolated in the form of the central control screen, but gradually transformed into multiple screens to assist driving tasks. For example, jaguar’s new XFL, audi’s new Q7 and other models all have four screens, including the central control screen, the dashboard display screen and two screens in the rear seat area. These four screens are not divided, but a whole that can display information together. The new seven is also based on four screens, adding a screen at the rear center armrest that can interact with the center panel [2].

Driving task should not only be undertaken by the driver alone, there is an objective need for social interaction in the car [3]. Contemporary car design must not only focus on technology supporting the driver and the driving task: it needs to create positive experiences for drivers and passengers alike [4]. Today a huge number of different driving assistance and navigation systems are available on the market. Often these systems fail to take into account the social nature and collaborative mechanism of driving [5]. Informing the development of future automotive user interface designs we need to develop a deeper understanding of collaboration in general. In addition, we need to develop an understanding of how, and in what way, other platforms (e.g., the mobile phones) are and will be used in combination with these systems while driving [6].

2 Related Work

The existing design and research mainly focus on the information exchange between cars and cars, or between drivers and cars, but often ignore the information exchange between drivers and passengers, the main driving pressure is still on the driver.

Case1: Information Interaction Between Drivers and Drivers

Due to an increasing level of automation whilst driving, users will gain additional spare time while travelling in the future. This allows passengers to indulge in non-driving activities, e.g. staying socially connected with their friends or entertaining themselves via mobile devices [7]. With everywhere available connectivity and the broad penetration of social network services, the relationship between drivers on the road may gain more transparency, enabling social information to pass through the steel shell of the cars and giving opportunities to reduce anonymity and strengthen empathy. The author utilized the “Four Side” communication model to describe different dimensions of information exchanging between drivers, which would be enhanced by the latest Vehicle to Vehicle communication technology [8]. Through this model, the driver can share music through adjacent vehicles. When the vehicle behind him needs to overtake due to an emergency (catching a plane), the driver can know the reason for speeding through the system, thus increasing the empathy between different drivers on the road.

Case2: Information Interaction Between Drivers and Pedestrians

One of the major challenges that autonomous cars are facing today is driving in urban environments. To make it a reality, autonomous vehicles require the ability to communicate with other road users and understand their intentions. Such interactions are essential between the vehicles and pedestrians as the most vulnerable road users [9].

In this case, the author designed a novel interface called “eyes on a car” to establish eye contact between self-driving cars and pedestrians. The concept behind the “look at the car” interface is to convey the intention of the car to pedestrians as they prepare to cross the road. The results show that pedestrians can make the right decision to cross the road “faster” and feel “safer” if the approaching vehicle has eyes and is watching them. This study details a potential and reasonable way of communication between cars and pedestrians. It provides a new perspective for vehicle and pedestrian communication [10].

3 Concept

Based on our design motivation and previous research, we believe that the interaction design between drivers and passengers for the car should be based on the following concepts:

With the appearance of apple CarPlay, Android Auto, Mirrorlink, etc., the driver’s cellular phone and the on-board interconnect have become big trend [11]. Our design hopes to transform the existing equipment on the basis of not increasing the co-pilot learning cost, strengthen the co-pilot and driver communication, and connect the co-pilot mobile terminal platform with the vehicle center to participate in the driving information decision-making. Based on this concept, we believe that it is necessary to improve the existing vehicle-mounted systems and portable devices, rather than add new devices and interactions to increase the learning and visual pressure of drivers.

The interactive mode must adapt to the future development direction of L3, L4 autonomous driving, it is difficult for the traditional user interface to generate good user experience in this complex environment. Therefore, we should take the multi-mode interactive mode as the main design direction. This design assists the voice-based interface and visual feedback on the vehicle center [12]. The display should have the minimum amount of information for users to pay attention to [13]. It’s a natural way of interacting with a multi-screen interactive system to enhance the interaction between drivers and copilot, and the decision-making skills of passengers and drivers.

To sum up, this design needs to ensure that the driver’s main task is not disturbed, follow the logic of information display and interactive control, and meet the communication decision-making and social needs between the driver and the passengers in the car.

4 Prototype and Test

4.1 Experimental Environment

In order to simulate the main driving task, the user’s accuracy and response time to the driving task should also be recorded. On the established vehicle-mounted platform, we play traffic lights animations and other customized events on the screen to give the driver a sense of tension to simulate the real driving situation. When a red light (or any custom event) appears in the road animation on the screen, the driver needs to step on the pedal, and the system will record the response of the driver when he steps on the pedal. In this experiment, the occurrence of red light is random, and it is set that it must be stepped down within 3 s after the occurrence (Fig. 1).

Fig. 1.
figure 1

The experimental environment (Color figure online)

4.2 Experimental Process

  • Venue: A205 zhizao avenue

  • Number of persons: 2 (one acts as the driver and one acts as the co-pilot) * 10 pairs, a total of 20 persons

  • Test duration: 5 * 2 min for discussion in driving task, and 5 min for user interview after completion.

The wizard of oz approach is an effective way to examine the user’s interaction with the computer and facilitate rapid iterative development of dialogue phrasing and logic [14]. The phrase, the wizard of oz, has entered common usage in the field of experimental psychology in laboratory Settings, simulating the behavior of theoretical intelligent computer applications [15].

This experiment using The Wizard of Oz method, two subjects are played by a driver, one vice driving, background for making a scene in city design experiment, discussion topic for the pilot and the co-pilot common decision to select the destination nearby restaurants and dishes, and two tasks were five 5 min (respectively, two different interactive mode based on The same task), after The experiment, interview subjects for 5 min.

In the first task scenario, only voice interaction is adopted, and the co-pilot and the driver interact with each other by voice to select the dish result. In the second task scenario, a multi-mode interactive environment with multi-screen interaction and voice interaction is added. The co-pilot’s mobile phone is connected to the vehicle center. When the co-pilot queries the information, the information will be displayed synchronously on the vehicle center, and the driver will complete the task under the voice and image information.

5 Analysis

5.1 Data Analysis

We selected three sets of data of typical users’ simulated driving and drew them into line graphs. According to the data graph, in the 5 min driving experiment, user I encountered a red light [or any custom event] 99 times. The driving delay in the single interactive mode was 149.08 ms and 162.75 ms in the multi-mode interactive mode. User 2: there were 87 times of red light [or any custom event], and the average driving delay was 695.59 ms and 714.65 ms, respectively. User 3: a total of 90 times of red light [or any custom event] was encountered, and the average driving delay was 165.75 ms and 169.63 ms respectively.

By data we can see that different users of driving a delayed reaction rate is different, there are even bigger differences, but based on the same user interactive mode of different driving trend is the same, delayed reaction rate in multimodal interactive mode and the single interactive mode (voice interaction only) the reaction time of the mean difference of 37.15 ms, which can be concluded that compared the multimodal interaction mode for the user driving behavior will slow response speed, but the impact is very small, the amplitude. It will not interfere with the fluency of the driver’s main driving task and thus cause driving burden (Fig. 2).

Fig. 2.
figure 2

Typical data for three users in two different modes of interaction (Color figure online)

Through video analysis: orange-red is the time period for drivers to see the multi-screen interactive information. From the time dimension, we can see that in the multi-mode interactive mode, the viewing of the multi-screen interactive information by users is basically positively correlated with the driving delay.

Through 10 sets of experimental data, it can be found that when the driver and co-pilot enter the topic, at the beginning of the conversation, the driver will not pay special attention to visual interaction information, and the driver and co-pilot mainly focus on voice interaction. However, starting from the latter part of the test time, when the median total time length of the total experimental data is 37.4%, the visual interaction information of the vehicle center will become the main influence in the driving process, and arouse the interest of the driver and co-driver to participate in the decision-making together (starting from 37.4% of the total test time, the single time and total frequency of drivers staring at the screen are increasing). Then, when the decision is about to be made, at the end of the experiment (starts from 86.2%), the driver will change the driver’s attention from the multi-screen interactive information to the co-driver’s voice interaction.

At the same time, it can be concluded from the ten experimental videos that the frequency of using multi-mode interaction is very high. In task two, drivers spend more than two-thirds of their time watching interactive information on multiple screens to participate in decision-making.

By video analysis we can conclude that in the process of driving, the driver is tend to seek help from a co-pilot to common decision, in the whole topic discussion, the driver’s preferences trends are starting tendency voice interaction, and tendency of voice and the interaction of multiple screen information multimodal interaction, finally returned to voice interaction of volatility process.

Through the analysis, it is found that the multi-screen interactive information will have a major impact when the decision-making output is near, and the length and frequency of the driver’s staring at the screen will increase. Interestingly, when multimodal interaction is an important factor (the driver is immersed in the screen information), the voice communication between the driver and co-pilot will be reduced, but the decision-making speed will be increased (Fig. 3).

Fig. 3.
figure 3

Screen time analysis of typical users

5.2 User Interview

In the interview with drivers, more than 2/3 of the drivers said that the information can be more intuitively understood through the multi-mode interaction, and the co-pilot’s auxiliary participation can increase the decision-making efficiency. In the interview with the driver, more than 2/3 of the drivers said that the information can be more intuitively understood through the multi-mode interaction, and the co-pilot’s auxiliary participation can increase the decision-making efficiency. Compared with the single voice mode, the biggest advantage of multi-screen interaction for drivers is that they can let the co-pilot make decisions with more confidence.

When they feel that they need to make affirmation or choice in the task, they can more quickly participate in the task through the visual information in front of the screen to understand the progress. From this dimension, although the delay rate of driving will slightly increase, the overall driving efficiency will increase.

Compared with the co-pilot holding the mobile phone to search for information by himself, in the multi-screen interactive mode, the interaction between the driver and the co-driver will increase due to the information sharing. The main driver and co-driver can be more involved in the communication, increasing the sociality in the car and the information flow.

6 Conclusion

Based on the development of intelligent transportation, the design of vehicle-mounted system tends to be multi-screen, and the mode of information interaction tends to be multi-mode. From the perspective of user experience, this paper designs a task switching model of multi-driving task interaction to improve driving efficiency in multi-screen interaction and voice interaction mode. This system aims to strengthen the task flow between drivers and passengers, so that both drivers and passengers in the car can be more involved in the communication, increasing the social degree in the car, the information flow, and the efficiency of decision-making in the car. Although it has been verified by experiments that the response of driving tasks will be slightly delayed through multi-mode interaction, with the development of automatic driving, the response delay of driving tasks in the future can be ignored in the design.