Modeling microstructure of drivers’ task switching behavior

https://doi.org/10.1016/j.ijhcs.2018.12.007Get rights and content

Highlights

  • This study introduces a computational driver model that simulates drivers’ task switching behavior and associated eye glances.

  • The model shows joint influence of task structure, uncertainty about the roadway, and individual differences on glances.

  • The model can reproduce eye glances observed in an empirical study.

Abstract

A computational model is created to simulate drivers’ task switching behavior, or dynamic allocation of visual attention, while they are driving and engaging in a secondary task. The model takes the following into account: uncertainty about the roadway, task structure, and individual differences. The first factor, uncertainty, means a lack of information about the roadway that plays a significant role in switching attention back to the roadway. The second factor, task structure, reflects the driver's tendency to switch visual attention from the secondary task to the roadway at subtask boundaries as well as the tendency to continue to perform task to reach subtask boundaries. Lastly, the model considers the variability of performance speed across the driver population. The factors jointly influence the probability of switching attention in the model. We use the ABC-MCMC (Approximate Bayesian Computation – Markov Chain Monte Carlo) method to estimate model parameters that produce the microstructure of task switching. The fitted model generates glance patterns at a micro level that are consistent with those generated by participants in an experiment.

Introduction

Drivers frequently use in-vehicle infotainment systems and switch their attention back and forth between the in-vehicle interface and the roadway. Such interactions with in-vehicle systems provide entertainment or valuable driving-related information but often cause the driver to look away from the road for long periods of time, which can contribute to near crashes or crashes when these glances coincide with rapidly unfolding roadway events. It is often suggested that glances away from the road longer than two seconds are dangerous (Horrey and Wickens, 2007, Klauer et al., 2006). This raises the important question: what causes long glances away from the road? In this study, we aim to understand the microstructure of attention allocation and simulate it with a computational model to show the mechanisms that contribute to long glances.

Imagine a driver trying to control the in-vehicle climate with a touchscreen embedded in the center stack. Let us assume that the driver wants to touch the ‘temperature up’ button five times and ‘fan up’ button three times to set the climate. To maintain the stability of the vehicle and drive safely, the driver will have to switch her visual attention back to the roadway before pressing the buttons. In this case, it is likely that the driver switches her attention back to the roadway after five consecutive button presses for the temperature setting, stabilizes the vehicle in the lane and maintains headway distance with a vehicle ahead, and comes back to the fan speed setting to make three more button presses.

The reason this strategy is likely is because two sets of button presses (temperature and fan) are the two subtasks that comprise the task. By switching attention at the boundary of the subtask, or after completing one set of button presses, the driver does not need to keep the state of the task in mind (e.g., how many more times do I need to press the same button?) and must simply start the next subtask when returning to the task. Nevertheless, if the driver is driving on a curved road, which requires more frequent attention to the roadway and fine control of vehicle, the driver might switch attention even before completing the first subtask that consists of five consecutive button presses. A more cautious driver might switch attention multiple times before completing the first subtask. As such, the task switching decision is influenced by multiple factors: roadway demands, task structure (subtask boundaries), and individual differences.

In understanding the task switching decision, the first factor of interest is uncertainty about the roadway that accumulates while looking away from the road, which happens more quickly when the demand from the roadway is higher. Renninger et al. (2007) argued that people have to reduce uncertainty, where uncertainty is a consequence of variability in external stimulus and internal memory degradation (Sullivan et al., 2012). The classic approach to quantify uncertainty in driving is Senders’ model (1967). Senders suggested an entropy-based uncertainty model of visual information accrual during driving. According to this model, uncertainty accumulates while vision is occluded because of the loss of roadway information through memory decay or the dynamics of the roadway environment. The driver shifts attention back to the roadway when the accumulated uncertainty exceeds one's threshold. The glance patterns depend on how much information is present on the roadway segment, and how fast it is outdated as the vehicle travels within a dynamic environment or forgotten from the driver's memory while looking away. The Senders’ model has been widely accepted to date (e.g., Liang et al., 2012, Victor et al., 2015). The role of uncertainty is also implied in the studies on roadway demand and task switching. When roadway demand is high, drivers sometimes give up on a secondary task or postpone the task (Jamson and Merat, 2005). On curved roads glance duration is shorter than on straight roads (Tsimhoni and Green, 2001). When drivers were asked to drive with their vision occluded, the distances driven with eyes occluded varied with environmental demands of scenario (Kujala et al., 2015). Whereas subtask boundary is a binary element (i.e., a moment in a task is a boundary or non-boundary), uncertainty accumulates continuously and influences task switching decisions.

The second factor that affects glance patterns is the task structure, and specifically, subtask boundaries. People segment activities at either a fine- or coarse- grain when perceiving them (Zacks et al., 2001). In this study, we refer to this smaller chunk of a task as a “subtask”. The boundaries of subtasks can be signaled by change in task difficulty, different levels in task hierarchy, etc. (Iqbal and Bailey, 2006, Janssen et al., 2012). In general, subtask boundaries in a task influence task performance (e.g., Arnold et al., 2017). When engaged in multiple tasks concurrently and switching attention between them, people tend to switch attention at subtask boundaries because they are a natural breakpoint that reduces cognitive load (Iqbal and Bailey, 2006, Janssen et al., 2012, Payne et al., 2007, Salvucci and Bogunovich, 2010). At the same time, when people are close to completing a subtask, the approaching subtask boundary encourages people to continue to perform the task and reach at the subtask boundary rather than suspending the task before reaching the boundary. People sometimes fail to suspend a task, even when persistence has negative consequences (Fox and Huffman, 2002). This “task perseveration” happens when people become fixated on completing a task. Payne et al. (2007) explained this tendency from a foraging perspective; this study argues that people tend to stay on a task when more successes occur (i.e., they achieve their subgoals), but people are also highly likely to switch tasks after achieving a subgoal. In driving, people tend to continue to engage in the secondary task – the task on in-vehicle infotainment system – when they are near completion of the subtask (Lee et al., 2012). To our best knowledge, this characteristic has not been explored in modeling driver behavior.

The third factor of interest is individual differences. While driving context and task design influence drivers’ task switching behavior, individual drivers show different strategies in responding to these external factors. People have different experiences, capabilities, and personalities, and hence show different behavior and performance in the same context (e.g., Ophir et al., 2009). These differences are well documented in many empirical studies (Deery and Fildes, 1999, Furnham and Marks, 2013, Watson and Strayer, 2010, Zhang et al., 2005). For instance, high and low risk drivers show different behavior and adaptation to driving situation (Peng and Boyle, 2015). Higher-risk drivers tended adapt to in-vehicle technologies in a negative manner and exhibit longer glances away from the road. In the similar vein, Kujala et al. (2015) found that drivers have different preferences on the distance that they feel comfortable driving with their eyes occluded, which implies that drivers have different tolerance in the duration they take eyes off the road in a certain driving context. However, the topic of individual differences has been scarcely mentioned in literature on driver behavior modeling; instead, drivers with different traits have been amalgamated into a single representative driver in most modeling studies. Because the tail of the glance distribution is more important in studying driver distraction due to its correspondence to longer glances, individual differences need to be recognized throughout the analysis of glances at subtask boundaries.

All three factors mentioned above determine how a driver switches attention while being engaged in a secondary task. One way to understand how these factors jointly influence such behavior is by building a mathematical model that simplifies such complex phenomena and illustrates relationships between variables involved. Although the factors mentioned above are not comprehensive, by simplifying complicated observations to a set of logical relationships, models allow researchers not only to interpret what has happened, but also to predict what will happen with given conditions, without direct observations. A dynamic driver behavior model can account for the time-dependent tendency to persist with a task. For an in-vehicle user interface design project with multiple design alternatives, a driver model can provide a method to identify design alternatives that produce long glances early in the process.

There are driver models created to explain and simulate driver glance behavior (e.g., Pettitt et al., 2007, Salvucci, 2006). More specifically, Salvucci and Kujala (2016) formalized a computational model that balances the subtask structure and uncertainty about the driving environment. The model was able to produce glance behavior that resembles that of real drivers. We further took a probabilistic approach to better account for dynamic nature of human behavior. When categorized largely as deterministic or probabilistic models, deterministic models yield a single output with given conditions, and probabilistic models yield a distribution of outputs even with the same initial condition. In modeling human behavior, the former gives an approximation of the most likely outcome, and the latter gives a distribution of likely outcomes. When system performance depends on the distribution of outcomes (e.g., longer glances) rather than just the most likely outcome, probabilistic modeling is more appropriate. Deterministic modeling may introduce variability by adding errors to measures, but, in this case, variability is added as noise rather than systematic variation in behavior.

This paper introduces a dynamic stochastic driver model, where the task switching decision of drivers depends on the current state and the associated probabilities of the state. We considered the three factors described above: task structure, uncertainty, and individual differences. Such a driver model generates a range of behaviors that the driver population may generate, rather than the behavior of a single, average driver. This model also reflects the distribution of behaviors that a single driver might generate. The model is built upon the glance patterns observed in a simulator experiment, where drivers were exposed to two different types of tasks in the same driving scenario: text reading and text entry. Estimating parameters of a dynamic and stochastic model requires techniques that go beyond the typical least-squares estimation approaches. In this study, we use Approximate Bayesian Computation, which is introduced in Section 3.2, and the parameter sets that fit each participant are found to simulate and validate the glance patterns (Toni et al., 2009). Below we describe the experiment and the model in separate sections. The parameter fitting method will be introduced in the model section as well. In the Experiment section, we report glances at the subtask boundaries that are compared to model predictions.

Section snippets

Participants

48 participants from four different age groups (18–24, 25–39, 40–54, and 55+ years old, 12 from each) with an equal number of males and females completed the experiment. Using the inclusion/exclusion criteria of the NHTSA Distraction Guidelines, each participant met the following requirements:

  • Is in good health;

  • Is an active driver with a valid US driver's license;

  • Drives a minimum of 3000 miles per year;

  • Is in the age range of 18–75 years of age;

  • Is comfortable using a computer and touch screen;

  • Is

Driver task switching model

A probabilistic model of driver behavior was created to replicate dynamic task switching behavior. There were four components that contributed to the probability of switching attention to the roadway: one representing uncertainty, two representing subtask boundaries, and one representing individual differences. Regarding the subtask boundary components, one was used to symbolize the higher switching tendency at boundaries, and the other was used to symbolize the reduced switching tendency on

Discussion

The current model was able to simulate the dynamic glance behavior of drivers and the microstructure of such behavior. This is different from static linear models that can be fit with least squared or maximum likelihood techniques and generate only summary measure of driver behavior, such as mean glance duration or maximum glance duration. Because the modeled driver behaved in a probabilistic way, it introduced natural variability and allowed us to observe naturally occurring outlier cases and

Acknowledgments

Data used in this paper is derived from research supported by the U.S. DOT – National Highway Traffic Safety Administration under Contract No. DTNH22-11-D-00237. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the NHTSA or the U.S. DOT. We would like to acknowledge the contribution made by Madeleine Gibson who helped the data collection and reduction.

Ja Young Lee is a PhD student in the Department of Industrial and Systems Engineering at University of Wisconsin-Madison with focus on Human Factors and Ergonomics. She obtained an MS degree in Industrial Engineering at North Carolina State University in 2014 and a BS degree in Industrial Engineering at Seoul National University in 2012.

References (44)

  • C.L. Dudek et al.

    Flashing message features on changeable message signs

    Transp. Res. Rec. J. Transp. Res. Board

    (2006)
  • C.L. Dudek et al.

    License plate and telephone numbers in changeable message sign ABMER alert messages

    Transp. Res. Rec. J. Transp. Res. Board

    (2007)
  • S. Fox et al.

    Escalation behavior as a specific case of goal-directed activity: a persistence paradigm

    Basic Appl. Soc. Psych.

    (2002)
  • A. Furnham et al.

    Tolerance of ambiguity: a review of the recent literature

    Psychology

    (2013)
  • F. Hartig et al.

    Statistical inference for stochastic simulation models - theory and application

    Ecol. Lett.

    (2011)
  • W. Horrey et al.

    In-vehicle glance duration: distributions, tails, and model of crash risk

    Transp. Res. Rec. J. Transp. Res. Board

    (2007)
  • S.T. Iqbal et al.

    Leveraging characteristics of task structure to predict the cost of interruption

  • A.H. Jamson et al.

    Surrogate in-vehicle information systems and driver behaviour: effects of visual and cognitive load in simulated rural driving

    Transp. Res. Part F Traffic Psychol. Behav.

    (2005)
  • C.P. Janssen et al.

    Natural break points: the influence of priorities and cognitive and motor cues on dual-task interleaving

    J. Cogn. Eng. Decis. Mak.

    (2012)
  • E.B. Khosroshahi et al.

    Capturing the effects of moderate fatigue on driver performance

  • S.G. Klauer et al.

    The Impact of Driver Inattention On Near Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data, Analysis

    (2006)
  • T. Kujala et al.

    The attentional demand of automobile driving revisited: occlusion distance as a function of task-relevant event density in realistic driving scenarios

    Hum. Factors

    (2015)
  • Cited by (17)

    • The role of individual differences in human-automated vehicle interaction

      2024, International Journal of Human Computer Studies
    • Modeling the effects of perceived intuitiveness and urgency of various auditory warnings on driver takeover performance in automated vehicles

      2022, Transportation Research Part F: Traffic Psychology and Behaviour
      Citation Excerpt :

      Next, the cost for task-switching in automated driving should be discovered and applied to the model. Several studies were conducted on task-switching, but most of them (1) focused on dual tasks (Jeong & Liu, 2017; Lee & Lee, 2019), (2) did not mathematically model the behavior (Markkula et al., 2018), or (3) computationally modeled task-switching behavior but did not consider the context in automated vehicles (Altmann & Gray, 2008; Brown et al., 2007; Gilbert & Shallice, 2002; Herd et al., 2014; Meiran, 2000; Von der Gablentz et al., 2015). Finally, the entire takeover transition might be more systematically captured when considering the concept of driver availability by assessing the driver state before and during the transition process (e.g., Marberger, 2017).

    View all citing articles on Scopus

    Ja Young Lee is a PhD student in the Department of Industrial and Systems Engineering at University of Wisconsin-Madison with focus on Human Factors and Ergonomics. She obtained an MS degree in Industrial Engineering at North Carolina State University in 2014 and a BS degree in Industrial Engineering at Seoul National University in 2012.

    John D. Lee is the Emerson Electric professor in the Department of Industrial and Systems Engineering at the University of Wisconsin, Madison and director of the Cognitive Systems Laboratory. His research focuses on the safety and acceptance of complex human-machine systems by considering how technology mediates attention. He is a coauthor of the textbook, Designing for People: An Introduction to Human Factors Engineering, and he helped to edit The Oxford Handbook of Cognitive Engineering.

    We have no conflicts of interest to disclose.

    View full text