A proactive decision support system for predicting traffic crash events: A critical analysis of imbalanced class distribution

Abstract

Real-time crash prediction plays a key role in enhancing traffic safety as well as mitigating disruptions to road users. The further improvements of predictability require the systemic analysis of crash likelihood within the driver–vehicle–environment triptych. This study presents a proactive decision support system that can predict crash events based on vehicle kinematics, driver inputs, roadway geometric features and real-time weather data. Modeling approaches that rely on Random forest, Support Vector Machine and Multilayer Perceptron machine learning techniques were applied to establish efficient crash predictions. Moreover, crash events are generally unexpected and occur rarely, thus classification results can yield deceivingly high prediction performance which are usually driven by the majority class at the expense of having poor performance on the crucial minority class. Therefore, this paper attempts to add to the current knowledge by investigating crash likelihood based on compared different data balancing techniques to improve the predictive performance through three balancing techniques: over-sampling, under-sampling and synthetic minority over-sampling (SMOTE). The highest performances have been acquired using SMOTE strategy as MLP achieved a 94.5% precision, 94.2% f1-score, 93.7% AUC and 95.3% recall, while SVM achieved a 91.5% g-mean. Furthermore, results indicated that more than 62% of total crashes have been reported in downhills and curved downhills, and 44% of all crash instances have been reported during both snow and rain weather patterns. Overall, the findings highlighted the significance of the explanatory variables associated with potential crash events and can suggest to decision-makers a safe and credible system for enhancing traffic safety.

Introduction

Traffic accidents are one of the most serious and threatening problems that encounters societies nowadays, having a detrimental influence both on a person and community level, resulting in many health issues, economic losses and fatalities. The World Health Organization [1] reports that 1.35 million people die in road traffic crashes every year, and a further 20–50 million are injured or disabled worldwide. The effects of road geometry and weather conditions have been identified to have major impacts on the identification of crash events [2], [3], [4]. On the other hand, related research depicted the vital implication of driver input responses and vehicle kinematics on safe driving and on the identification of crash and near-crash events [5], [6], [7]. However, little effort has been made to quantify the effects of real-time data combining weather conditions and road geometry along with driving maneuvering inputs and vehicle kinematics on predicting the occurrence of crash collisions. Therefore, a reliable system for crash collisions forecasting and proactive safety analysis is undeniably of great interest and necessity.

Crash analysis is a complex mechanism, affected by various contributing factors including the vehicle telemetry, driver state and environmental factors [8], [9]. While the related research have investigated unsafe driving behaviors in an attempt to characterize traffic crashes and develop real-time road management policies [6], [10], [11], exploring the impact of real time information acquired from the driver, vehicle, weather and road geometry scheme is relatively limited. Weather status was found to cause that more than 1.25 million accidents (21% of all vehicle crashes), leading to about 418,000 injuries (19% of crash injuries), and nearly 5000 casualties (16% of all casualties) [12], however, it was found that most of the literature that adopted weather variables in crash assessment used data obtained from police crash reports which could be susceptible to inaccuracies as the reported conditions may be what were observed by the person filling the crash report and not the effective weather status at the time of accident [5], [13]. Also, driver input action as well as vehicle kinematics have proven to have an essential impact on the recognition of crash and near-crash events [6], [14]. As concerns the roadway geometry, relevant scholars studied the effects of route geometric characteristics and how they lead to a substantial change of the driving behavior [4], [15]. In this study, real-time data were gathered using a driving simulator; transportation research has been actively adopting driving simulation experiences as they are much safer, with the major advantage of possessing full empirical control over conditions and the capacity to explore multiple design structures [16]. The adopted route layout included various geometry types such as uphills, downhills, curves, straight lines and roundabout stretches, whereas simulations were conducted during three adverse weather covariates namely fog, rain and snow seasons. The driver input responses (e.g. pedal positions and wheel angles) along with vehicle kinematics (e.g. speed and Time-To-Collision) were systematically recorded during the trials and preprocessed in order to identify the best precursors for crash events evaluation.

In crash prediction analysis, traditional statistical learning-based techniques such as logistic regression [10], quantile regression [17] and discriminant analysis [18] have been largely utilized. However, statistical models for crash prediction frequently suffer from poor data quality and require great deal of historical data and provide unsatisfying results when treating features with a high number of categories [19], [20]. Conversely, machine learning (ML)-based models have proven to supersede statistical analysis in predicting forthcoming events and have reported satisfying results in many transportation systems [21], [22], [23]. The significant interests of ML models can be characterized by (i) their autonomously surmounting major non-linear problems using datasets from multiple sources; (ii) their ability to easily incorporate newly data in an attempt to improve estimation performance, (iii) and their predictive and explanatory ability through the extraction of rules. The Support Vector Machines (SVM), Random Forest (RF) and Multilayer Perceptron (MLP) are ones of the most substantial machine learning techniques that have been used for crash events prediction [24], [25], [26], [27], [28]. Endorsing SVM in assessing safety performance measures for vehicle crashes depicted a good handling of small data sizes, with a great capability in producing fewer over-fitting issue and better generalization abilities [29]. At the same time, relevant scholars have affirmed the performance of RF in several fields as it Random Forest has been proven to reduce variance compared to a single decision tree, and it is also robust to outliers and missing values [30]. On the other hand, MLP gained its popularity owing to their excellence in various complex tasks by learning data representations in both supervised and unsupervised settings along with parallel processing, fault tolerance, and the efficiency to generalize to unseen data samples using hierarchical representations [31].

Another key role determinant in the prediction of crash events is the ratio of crash and non-crash instances in the dataset. In road accidents related observations, there usually are relatively fewer crash data points compared to no accidents’ samples. Numerous researchers embrace the conventional proportion of accepting 4 non-accident cases for each accident case [29], [32], [33]. However, this is likely to result in imbalances as there would be a bias toward the majority class since that predictive learners prioritize the label with the greater number of instances leading to an over-prediction of this class [5], [34]. Conventionally, oversampling and undersampling are two essential techniques that are largely utilized to address class-imbalance issue. Oversampling approaches prevent data loss by focusing on duplicating the samples of the minority class. In contrast, undersampling strategies attempt to balance the ratio of the classes by eliminating data points from the majority class. The Synthetic Minority Oversampling Technique (SMOTE), judged as one of the most powerful re-sampling algorithms, was presented by Chawla et al. [35] to solve the imbalance issue by producing synthetic instances from the minor class. On the issue of skewed instances, SMOTE is capable of identifying similar but more specific sections in the feature dimension as the decision region for the minority class [36]. To enhance the performance of classification models with imbalanced dataset, variations of the resampling techniques along with the aforementioned classification models have been employed. Moreover, The variable extraction procedure has been conducted based on the widely employed using principal component analysis (PCA) which is a dimension reduction approach that finds a linear transformation of the input data points generating projections of the original features to a new variable space [37]; PCA has been proved to be one of the effective methods of dimensionality reduction as no prespecified structure is required for the input space while the amount of variance explained by each variable is maximized by the orthogonal (i.e., uncorrelated) parameters [38].

The objective if this study is three-fold: (1) to identify the strongest factors contributing to the likelihood of crash instances across various route geometries under adverse weather conditions using comprehensive real-time data, (2) to examine the effects of different combinations of the four adopted features’ categories – road geometric characteristics, weather patterns, vehicle telemetry and driver inputs – on the occurrence of crash events and (3) to develop multiple prediction models – RF, SVM and MLP – using different sampling strategies namely under-sampling, over-sampling and SMOTE to handle the data imbalance issue. We aimed to thoroughly comprehend the interrelationships among various combinations of the input space features, sampling techniques and modeling learners in crash events prediction. In order to reduce the sampling bias in splitting the data between training and testing for each model building process, we employed 10-fold cross validation. The remainder of this study is organized as follows. First, descriptions of the driving simulator and experimental protocol with data analysis are provided. Second, in the methodology section, the construction of modeling techniques is presented in details. Next, the results are reported and interpreted. Finally, conclusions with future scopes of the present study are offered.