Driver Behavior at Simulated Railroad Crossings

Abstract

Highway-rail grade crossing collisions and fatalities have been in decline for several decades, but a recent ‘plateau’ has spurred additional interest in novel safety research methods. With the support of Federal Railroad Administration (FRA), Michigan Tech researchers have performed a large-scale study that utilizes the SHRP2 Naturalistic Driving Study (NDS) data to analyze how various crossing warning devices affect driver behavior and to validate the driving simulation data. To this end, representative crossings from the NDS dataset were recreated in a driving simulator. This paper describes driver behavior at simulated rail crossings modeled after real world crossings included in the NDS dataset. Results suggest that drivers may not react properly to crossbucks and active warnings in the off position. Participants performed the safest behaviors in reaction to STOP signs. The majority of participants also reported an increase in vigilance and compliant behaviors after repeated exposure to RR crossings, which was supported by the results of a linear regression analysis. Participants used the presence of active RR warnings (in the off position) as a cue that there is no oncoming train and it is safe to cross without preparing to yield (operationalized as visually scanning for a train and active speed reduction). Drivers react the most appropriately to STOP signs, but it is unclear whether or not these behaviors would lead to a decrease in train-vehicle collisions.

1 Introduction

1.1 Literature Review

Historically, safety at highway-rail grade crossings (crossings) has been an area of special attention due to the gravity of accidents that occur at those locations. Crossing collisions and fatalities have been in decline for several decades, but a recent ‘plateau’ has spurred additional interest in novel safety research methods [1]. With support from the Federal Railroad Administration (FRA) we have conducted a large-scale study that utilizes the SHRP2 Naturalistic Driving Study (NDS) data to analyze how various crossing warning devices affect driver behavior and to validate the driving simulation data. To this end, representative crossings from the NDS dataset were recreated in a driving simulator using the National Advanced Driving Simulator (NADS) MiniSim simulation software and hardware. This paper describes driver behavior at simulated rail crossings modeled after real world crossings included in the NDS dataset.

The Naturalistic Driving Study (NDS) data with its five million trips provided an unprecedented opportunity to collect direct observations of drivers’ negotiating crossings, and to see how they react to the crossing environment, including the traffic control devices (warnings) currently used to warn drivers as they approach railway tracks [2]. The NDS data set includes trip data that can be used to observe driver behavior, and relate it to vehicle position during a traversal of a given crossing. Analysis of that behavior is complicated by the facts that traversals occur at different speeds, crossings are located in varying environments, and actions taken by drivers may occur in a different order at any given crossing.

Sign Types.

Most rail-related safety analyses start by categorizing crossings by warning sign type. Passive warnings such as crossbucks are the minimal level of warning at a crossing, are cheaper to install and require less maintenance than their active counterparts [3]. Of course, the downside of passive warnings is that they provide no information on the presence or absence of a train, only the presence of a crossing. Time-to-arrival judgment errors may cause incidents at passive crossings, but there are many more “failure to notice crossing/train” errors than at active crossings. Intervention methods such as increasing the saliency of the crossing and providing train-present information are expected to be as effective at passive crossings as active warnings with no physical barrier (flashing lights and bells) [4].

Non-compliant behavior such as gate-running is the biggest cause of accidents at crossings protected by active gates, while failure to detect the crossing or train, or at least, a poor time-to-arrival judgment is the largest cause of accidents at crossings protected by passive warning signs [4]. Because the nature of the breakdown in safety is due to different issues depending on the type of crossing, multiple countermeasures to increase safety must be developed to specifically target each issue independently and to promote respect for RR warnings in general.

Previous studies have shown that driver behavior varies even within types of passive crossings. For example, historical analyses of incident reports suggest that controlling for amount of traffic, STOP signs are associated with the highest rate of incidents [5]. However, previous simulation studies have shown that drivers are more likely to properly comply at STOP signs than at RR crossings with crossbucks only [6]. Drivers may not understand how to properly comply with crossbucks while STOP signs are much more common and explicit about behavior than crossbucks. The past inconsistent findings between real world and laboratory simulation driver behavior studies open up questions on “how should safety engineers use these conflicting findings to make better decisions about which type of warning to place at new or problematic RR crossings”? To answer these questions, we designed this line of research.

Naturalistic and Simulated Driving Studies.

Incident records in the FRA database often lack details, making interpretation of the data difficult. Each record includes quantitative and qualitative data that describe each incident, including a narrative section. Unfortunately, most of the narratives fail to describe the behavior of the driver and sometimes even conflict with the other information provided in the quantitative descriptions [4]. In addition to the lack of consistency and accurate or descriptive narratives, records of the signage type at crossings are often out of date. This suggests that some crossings are upgraded from passive to active warnings without the FRA crossing inventory being updated. However, the accident reports of incidents should show the existing traffic control devices at the time of the accident.

For the reasons listed above, it is worthwhile to search for more reliable datasets for rail safety analyses. We have approached the issue of unreliable or incomplete FRA incident records with two lines of research. The first line of research involves aggregating data from the SHRP2 NDS (naturalistic driving study) related to highway-rail grade crossings. This dataset includes camera, GPS, and vehicle sensor data describing how drivers actually behave in the real world.

Unfortunately, observational studies lack the control of laboratory simulations to allow for systematic manipulation of variables of interest. In short, the naturalistic driving data may not have the predictive validity to infer how driver behavior would change in response to novel warnings. To this end, the second line of research includes a high fidelity driving simulator to recreate and manipulate real-world crossing scenarios. The following sections will outline a study where participants are exposed to simulated crossings, modeled after real world crossings from the NDS data set. Behavioral data collected from the driving simulator is presented and preliminarily compared to the observed behaviors from the NDS.

Driving simulators have been increasing in fidelity over the previous decades. However, questions remain open about the predictive validity of laboratory simulations. Can the NDS dataset be used to validate the use of driving simulators to predict driver behavior to novel RR warnings? Can driving simulators be used to help understand and put into context the driving behaviors collected in observational studies such as the NDS? Can the combination of both laboratory simulation and naturalistic observation reveal new explainable patterns in driver behavior? Answers to these questions and more are the motivation for the following study.

2 Methods

2.1 Participants

Seventeen (M_age = 19.4, SD_age = 2.1, 11 male, 6 female) undergraduate participants were recruited from the local university participant pool in exchange for course credit. All participants had at least three years of driving experience. Fifteen (88%) of the participants reported encountering a railroad crossing at least once per month on average. None of the participants reported any simulator induced motion sickness.

2.2 Apparatus (Driving Simulator)

For this study, we used the NADS simulator running the MiniSim version 2.2 software. The simulation software runs on a single computer, running Microsoft Windows 10 Pro on an Intel Core i7 processor, 3.07 GHz and 12 GB of RAM, and relays sound through a 2.1 audio system. Three Panasonic TH- 42PH2014 42″ plasma displays with a 1280 × 800 resolution each allow for a 130° field of view in front of the seated participant. The center monitor is 28 in. from the center of the steering wheel and the left and right monitors are 37 in. from the center of the steering wheel. The MiniSim also includes a real steering wheel, adjustable car seat, gear-shift, and gas and brake pedals, as well as a Toshiba Ltd. WXGA TFT LCD monitor with a 1280 × 800 resolution to display the speedometer, etc. Environmental sound effects are also played through two embedded speakers. These sounds included engine noise, brake screech, turn indicators, collisions, auditory alerts, etc.

A single webcam was positioned to capture the visual scanning behavior of the participant’s face while in the simulator. This video feed was recorded and integrated with screen capture recordings of the driving simulator’s visuals and instrument panel. Open Broadcast Software (OBS) was used to capture and integrate the visuals from the driving simulator and webcam into one video file.

2.3 Stimuli (Scenario Description)

The simulated scenarios were developed using the Tile Mosaic Tool (TMT) and Interactive Scenario Authoring Tool (ISAT), and presented to the participant using the NADS MiniSim version 2.2 software and hardware. Three scenarios were created, which will be referred to henceforth as scenarios “A”, “B”, or “C”.

All scenarios included moderate amounts of automated traffic driving in the other lane. All scenarios were comprised of three laps, and each lap had two identical crossings as far away from each other as possible (Figs. 1 and 2). Each lap took about 5 min for the participant to drive, resulting in around 15 min of drive time and 6 crossing events per condition. All scenarios could be described as “rural”, featuring very few intersections and buildings. All additional signage (Pre-warning signs and pavement markings) were included and placed at the appropriate locations for all simulated crossings.

Fig. 1.

The four types of simulated crossings (from top to bottom: Crossbuck, Gate, STOP, and Cantilever) featured in the scenarios.

Fig. 2.

Maps of scenario loops (left is condition A and B, right is condition C). RR crossings are located at the 12 and 6 o’clock positions of each loop.

All scenarios included one train present event on the 5^th of the six crossing events. The train event was triggered by the participant crossing over the pre-warning pavement markings, and the train would arrive at the crossing approximately 17 s after warning activation. We acknowledge that the Manual of Uniform Traffic Control Devices (MUTCD) recommends a minimum of 20 s between the activation of the traffic control device and the arrival of the train. However, based on pilot testing the 17 s warning time helped ensured that drivers were forced to make a decision whether or not to stop at the appropriate time on approach to the crossing.

Scenario A attempts to model crossing 505650D, a location in Sardinia, NY. This crossing includes active warning devices in the form of gates and blinking lights and was selected as a representative of one of the most common types of crossings represented in the NDS dataset. The crossing is perpendicular to the road and has an almost unobstructed view down the tracks except for a few trees. The entire scenario is a loop that takes about 5 min for the participants to drive one lap. Each lap includes two identical crossings modeled after the crossing described above. A yield intersection was included approximately 1/8^th miles before the first crossing warning in an attempt to control for the driver’s speed on approach. The lights, bell sounds, and gate arm are activated approximately 17 s before the arrival of the train at the intersection. The fifth crossing event of this scenario includes a train present event. All other crossing events contain no trains.

Scenario B is identical to scenario A, except for the type of warnings present at the highway-rail crossings. In scenario B the first crossing was changed to a passive crossbuck, and the other crossing was changed to a STOP sign and crossbuck. Scenario B is the only condition to incorporate two different types of crossings. The train present event (crossing event #5) in scenario B occurred at the crossbuck-guarded crossing.

Scenario C is modeled after crossing 621549W in Plant City, FL. This crossing has an overhead cantilever with flashing light warning, but no gate arm. Roughly, ½ a mile of road was recreated before and after the crossing to model the real world environment using Google Maps’ street view as a reference. This was necessary to reflect the uniqueness of this intersection. The view down one direction of the track is completely obstructed by buildings, and the track curves behind trees obstructing the view down the other direction of the track. On the 5^th crossing event, the train approaches from behind the building. The lights and bell sounds are activated approximately 17 s before the arrival of the train at the intersection.

2.4 Design and Procedure

Each participant drove through all three conditions to set up a within-subjects comparison. The order of presentation was fully counterbalanced to wash out its’ effect over the whole group of participants. Participants were allowed to practice driving for 1 min to adjust to the idiosyncrasies of the driving simulator. After all three conditions, participants completed an online survey before being released. The survey questions included:

• “How many times do you encounter a train crossing per month on average?”

• “How realistic were the scenarios/your driving behavior?”

• “How did your behavior at railroad crossings change over the course of this experiment?”, and

• “What should/would you do when approaching this sign?” (presented with pictures of the different types of railroad warnings depicted in the simulation).

Driver Behavior Coding Scheme.

Driver behavior was operationalized as a score ranging from 0–4 using a novel coding scheme developed in a previous study (Table 1) [6]. The coding scheme awards 1 point (max of 2) for each direction looked at the appropriate time. This visual scanning behavior was only counted if the behavior was performed after passing the pre-warning traffic sign, but early enough to allow for appropriate braking time if an oncoming train was spotted. The driving simulator also recorded pedal depression and vehicle speed. One point was awarded if the participant completely removed their foot off the accelerator pedal (an indication of coasting), and another point was awarded if the participant placed any pressure on the brake pedal (an indication of active speed reduction). Again, points were only awarded if the behavior occurred at the appropriate time (after passing the pre-warning sign, but early enough to react appropriately to an oncoming train).

Table 1. Behavior score coding scheme

It is necessary to modify the behavior score coding scheme to accommodate for the intended behaviors at STOP protected crossings. For STOP protected railroad crossings, one point is given for active speed reduction (no force on the accelerator pedal), and another is given for coming to a complete stop (when vehicle speed is virtually 0 mph). Visual glances down either side of the track are only awarded points if they occur while the vehicle is completely stopped. Theoretically, more behaviors are required to achieve the same 4 point score at STOP protected rail crossings than for all other warning types. There was no modification to the driver behavior coding scheme for train present events (event #5 in each condition). The presence of the train (or activation of active RR warnings) results in inflated scores compared to train-absent events, but its effect is washed out in within-subject comparisons. One researcher applied the coding scheme on the collected videos to create the visual scanning portion of the behavior score. “If” statements for pedal depression by event were coded in Rstudio to create the other half the behavior score. These two data sets were then combined (summed together) to create the total 0–4 behavior score.

3 Results

Due to experimenter and software errors the data for four participants was dropped because it was incomplete. As a reminder the behavior score combines visual scanning behavior (coded from the videos) and vehicle data (pedal depression and vehicle speed) collected by the simulator. A score of 0 indicates that the driver virtually ignored the crossing, and a score of 4 indicates that the driver responded appropriately in relation to the crossing.

A repeated measures ANOVA was conducted to observe the effect condition (warning type) has on behavior scores. Results suggest condition (warning type) has no significant effect on participant’s behavior scores (F(1.1.37, 9.096) = 4.654, p = .056) (Table 2).

Table 2. Tests of within-subjects effects ANOVA.

However, given that the results are approaching significance, post hoc pairwise comparisons with a Least Significance Difference (LSD) adjustment were conducted to compare individual conditions against each other. The post hoc pairwise comparisons suggest condition B (passive STOP and crossbuck) behavior scores are significantly higher than condition C (active cantilever) scores (p = .027) (Table 3).

Table 3. Post hoc pairwise comparisons between all conditions.

Analyzing participant’s behavioral data chronologically also allows for extrapolating how driver behavior can change after repeated exposure. Note how behavior scores increase at the train present event 5 in both active warning conditions A and C. In condition B (green line in Fig. 3), all of the even numbered events (2, 4, & 6) have STOP protected rail crossings, while all odd events (1, 3 & 5) have crossbuck protected rail crossings.

Fig. 3.

Mean behavior score by event, grouped by condition (warning type). Error bars represent standard error. (Color figure online)

Figure 4 depicts mean compliance scores by event experienced chronologically in the experimental session. This is the truest representation of how behaviors change after repeated exposures to railroad crossings, regardless of warning type. A simple linear regression was calculated to predict behavior score based on event number. A significant regression equation was found (F(1,229) = 10.22, p = .0016, with an R² of .042. The model predicts that behavior scores increase on average .05 points for each additional exposure to a RR crossing.

Fig. 4.

Behavior scores over the entire course of the experiment (chronologically).

Additionally, the very slight increase in behavior score over time is backed up by the responses in the post experiment survey. Thirteen (75%) of the participants reported becoming more cautious at following events after experiencing a train present event. Alternatively, one of the participants reported becoming less cautious over time because they “trusted the signs”.

From the stacked bar chart (Fig. 5), we can see the majority of points awarded in conditions A and C are from visual scanning behaviors. This suggests that participants are more likely to visually scan for a train than actively reduce vehicle speed on approach to active (but OFF) rail warnings. Figure 5 also shows how participants are more likely to perform “yield behaviors” such as coasting or actively reducing speed for passive warnings than for active (but OFF) warnings.

Fig. 5.

Stacked behavior score by condition, split by visual scanning and pedal depression behavior. Scores are averaged across all participants by condition.

4 Discussion

Overall, results have followed patterns seen in previous driving simulator experiments. Behaviors scores are at their highest at STOP protected rail crossings, and the lowest for active (but OFF) warnings. It is promising that behavior scores associated with train present crossing (event #5) are consistently high across all warning types. This is particularly interesting in condition B, where the train present event occurred at a passive crossbuck protected crossing. In the simulation, the train was completely silent due to software limitations. Participants were able to identify an oncoming train during their visual scans on approach even without the aid of an active warning.

Visualizing the data chronologically does provide some evidence that participants become slightly more vigilant over time after repeated exposures to railroad crossings (Fig. 4). Perhaps the consistent presence of a train on event 5 in each condition was the main driver of this trend. Similar chronological analysis will be performed with the NDS data which may shed light on how behavior changes over time in naturalistic settings.

5 Conclusion and Future Works

It is useful to mention that our behavior score coding scheme cannot predict likelihood of an incident. It is possible to score low while completely avoiding the train. It is also possible to score high and still collide with the oncoming train. The fact that the train is technically in the driver’s field of view does not guarantee the driver will notice the train, and understand how to behave in order to safely avoid an incident.

In conclusion, the data from this study suggest that participants in a driving simulator generally understand that the burden of safety is on the driver for passive crossings, but not for active crossings. Participants trust active warnings to warn drivers of approaching trains, and the OFF status of the warning device is used as a cue that there is no hazard. Unfortunately, active warnings are not immune to malfunctions, which could lead to deadly consequences if drivers are of this mindset in the real world.

Future works include comparisons between the driving simulator data and NDS data. If similar patterns are observed in both data sets, more effort could be spent conducting simulator studies as a valid representation of real world driving behaviors.