STMAG: A spatial-temporal mixed attention graph-based convolution model for multi-data flow safety prediction

doi:10.1016/j.ins.2020.03.040

Information Sciences

Volume 525, July 2020, Pages 16-36

https://doi.org/10.1016/j.ins.2020.03.040 Get rights and content

Highlights

•
We present a safety prediction model called the Spatial-Temporal Mixed Attention Graph-based Convolution model (STMAG).
•
A case study on the implementation of this model in traffic safety prediction is given as example.
•
The effectiveness of STMAG has been verified by our theoretical analysis and experimental results.

Abstract

Spatiotemporal safety forecasting has various applications in the neuroscience, climate and transportation domains. It is challenging due to (1) the complex spatial dependency on networks, (2) non-linear temporal dynamics with changing conditions and (3) the inherent difficulty of long-term forecasting. To address these challenges, a safety prediction model called the Spatial-Temporal Mixed Attention Graph-based Convolution model (STMAG) is proposed. Specifically, STMAG captures spatial dependency using graph convolutional networks (GCN), and temporal dependency using the sequence-to-sequence (Seq2Seq) architecture with the mixed attention mechanisms. A case study on the implementation of this model in traffic safety prediction is given as an example. Traffic safety forecasting is one canonical example of such a learning task, which is also a crucial problem to improving transportation and public safety. A number of detailed features (such as vehicle type, braking state, whether changing lanes or not) and exogenous variables (such as weather, time and road condition) are extracted from our big datasets. Finally, we conduct extensive experiments to evaluate the STMAG framework on real-world large-scale road network traffic datasets. Extensive experiments on our dataset show that the STMAG framework makes reasonably accurate predictions and significantly improves the prediction accuracy over baseline approaches.

Introduction

Safety is an important consideration in every workplace and has a significant impact on human life [25]. Many accidents occur every year in China, especially in recent years, where many major accidents have had an extremely adverse impact on China’s economic and social development, seriously disrupting social order. Therefore, safety prediction is a common thread throughout every workplace. It is necessary to reduce the impacts and take action against hazards that may occur in different scenarios by performing safety prediction analysis [2].

Every activity has certain inherent potential for accidents [35]. The root cause of the vast majority of accidents is the unsafe behavior of human beings [42]. Therefore, it is particularly important to monitor unsafe behavior in an accident safety prediction system. There are different unsafe behaviors in different scenarios, such as walking in the wrong direction, parking in a restricted area and entering a forbidden area.

A safety prediction system provides timely prediction and warning of an impending accident. It can take countermeasures to address the danger and prevent the occurrence of the accident [21]. At present, however, a majority of industries have not established a scientific and effective accident safety prediction system. Some existing early warning systems lack online dynamic monitoring technology. Accident safety prediction is overly dependent on artificial experience and it is hard to detect an impending accident. Therefore, accidents cannot be averted in their initial stages.

Reliable prediction models are important for a variety of applications. Safety prediction is based on the potential information and features extracted from the historical data flow. In the process of predicting the data flow, the most crucial question is to think about how to extract the characteristics of Multi-data flow completely. Moreover, the features of nonlinearity and time sequence in the multi-data flow make the prediction difficulty significantly increased.

With the development of the collection devices, they are deployed in various areas, providing a solid data foundation for safety prediction [24]. Early, time series analysis models are employed for prediction problems. However, it is difficult for them to handle the unstable and nonlinear data in reality. Subsequently, traditional machine learning approaches are developed to model more complex data, but they are still hard to simultaneously consider the spatial-temporal correlations of high-dimensional data. In addition, the prediction performances of such methods depend heavily on feature engineering, which often requires a large number of experiences from experts in the corresponding fields. In recent years, many researchers use deep learning methods to process high-dimensional spatial-temporal data, that is, convolutional neural networks (CNN) [26] are employed to effectively extract the spatial features of grid-based data; GCN are adopted for describing spatial correlation of graph-based data. However, these methods still fail to simultaneously model the spatial-temporal features and dynamic correlations of Multi-data flow.

To cope with the above challenges, we propose a novel deep learning model: Spatial-Temporal Mixed Attention Graph-based Convolution model (STMAG). The core goal of our proposed STMAG model is to predict the safety status of a system in advance, based on dynamic multiple data streams collected from video monitoring and other devices and to actively defend against hidden dangers in the system. This model can provide new ideological guidance and strong technical support for an accident safety prediction system. This paper will use traffic safety prediction as a starting point to explore the effectiveness of this model.

In this paper, the safety prediction framework STMAG is used in the field of transportation. Traffic accidents cause a huge number of casualties and property loss. Reducing the number of traffic accidents is a crucial social problem. We extract the spatial features based on GCN and incorporate the temporal features with the mixed attention mechanism in the model to better capture the temporal trends and spatial heterogeneity of the data.

As shown in Fig. 1, the host vehicle is affected by the state of the preceding vehicle and external factors such as the weather and time. The feature extraction from the video that is captured by the host vehicle’s dashboard camera includes information such as the type of preceding vehicle, lane detection and brake light detection, etc., and the weather conditions are obtained through the network interface provided by the Bureau of Meteorology. These features are simultaneously imported into our model to obtain the prediction results. If there is a danger, the driver of the host vehicle will be alerted in order to avoid an accident.

Our contributions are summarized as follows:

1.
The STMAG model is applied to the spatio-temporal prediction tasks, which combines the GCN and GRU. Specifically, the GCN is used to capture the topological structure of the network to model spatial dependence. The GRU is applied to capture the dynamic change of spatiotemporal data to model temporal dependence.
2.
It is common sense that not all nodes and variables contribute to spatio-temporal prediction tasks equally. To incorporate the different importance of the variables in each node over time, the mixed temporal and variable attention mechanism is introduced into the safety prediction model.
3.
Finally, we evaluate the effectiveness and better understanding of the STMAG model in a transportation field. The results show that our approach has superiority in traffic forecasting.

The rest of this paper is organized as follows: in Section 2, we introduce the related work. Section 3 presents the STMAG safety prediction model. In Sections 4, a case study on the application of this model to traffic safety prediction is given. Our experiment results and analysis are shown in Section 5. Finally, we conclude this paper in Section 6.

Section snippets

Related work

Safety prediction has the goal of preventing accidents or reducing their incidence rates and impacts[6]. There are many factors which can influence the occurrence of accidents. Under certain conditions, unsafe human behavior and an unsafe state of things can together lead to accidents. Probable accidents, fatal faults and preventative measures should be identified in advance to reduce the frequencies and impacts of accidents to the lowest possible levels. Safety prediction measures have been

Preliminaries

In this subsection,we explain the definitions and notations of the variables used herein.The key notations employed in this paper are described in Table 1. The definition of the network topology:

Definition 1

The network G. We use an undirected graph $G = (V, E)$ to represent the topological structure of the network, where $V = {v_{1}, v_{2}, \dots, v_{N}}$ is a set of nodes, N is the number of nodes, each node has P attributes, and E is a set of edges. The adjacency matrix A represents the connection between nodes, $A \in R^{N \times N}$ . The

A case study on traffic safety prediction based on the STMAG model

To test the availability and better understand the STMAG model, a safety prediction task is carried out in a traffic area. A traffic safety prediction task requires the support of an effective model, and our proposed STMAG model is suitable for this scenario. In order to realize the real-time detection of the abnormal state, videos from the dashboard camera and traffic input data extract the features of the preceding vehicle and external variables as anomaly detection sequences to characterize

Experiment

In this section, we conduct experiments on real datasets. We undertake a comprehensive quantitative evaluation by comparing our method with other baselines approaches.

Conclusions

Safety prediction is an important and challenging task in many domains as it is affected by many complex factors, such as spatiotemporal dependencies as well as external influences. In this paper, a new safety prediction analysis and monitoring method (STMAG) is developed to identify and prevent accidents, and the proposed method is applied to traffic safety prediction.

In particular, we propose to adopt GCN to capture graph-based spatial dependencies and employ Seq2Seq architecture to obtain

Funding

This work was supported by the Shanghai Key Science and Technology Project (19DZ1208903); National Natural Science Foundation of China (Grant nos. 61572325 and 60970012); Ministry of Education Doctoral Fund of Ph.D. Supervisor of China (Grant no. 20113120110008); Shanghai Key Science and Technology Project in Information Technology Field (Grant nos. 14511107902 and 16DZ1203603); Shanghai Leading Academic Discipline Project (No. XTKX2012); Shanghai Engineering Research Center Project (Nos.

Author contributions

Qingkui Chen proposed the main idea of this paper. Qingkui Chen and Jingjuan Wang designed the spatial-temporal mixed attention graph-based convolution model for multi-data flow safety prediction. Qingkui Chen, Jingjuan Wang and Huilin Gong conceived and designed the experiments. Jingjuan Wang performed the experiments and analyzed the data and the results. Qingkui Chen served as advisors and made suggestions on the experiment evaluation. Jingjuan Wang wrote the paper. All authors have read and

Declaration of Competing Interest

There are no conflicts of interest.

Acknowledgments

The authors would like to acknowledge the support from the Flow Computing Laboratory at University of Shanghai for Science and Technology.

References (42)

L.-Y. Chang et al.
Data mining of tree-based models to analyze freeway accident frequency
J. Saf. Res.
(2005)
D.K. Hammond et al.
Wavelets on graphs via spectral graph theory
Appl. Comput. Harmon. Anal.
(2011)
W.-C. Hong
Traffic flow forecasting by seasonal SVR with chaotic simulated annealing algorithm
Neurocomputing
(2011)
L. Lin et al.
A novel variable selection method based on frequent pattern tree for real-time traffic accident risk prediction
Transp. Res. Part C
(2015)
Q.-H. Nguyen et al.
Car-to-pedestrian communication with MEC-support for adaptive safety of vulnerable road users
Comput. Commun.
(2020)
S. Qiu et al.
A valuation-based system approach for risk assessment of belief rule-based expert systems
Inf. Sci.
(2018)
D. Xia et al.
A distributed spatial–temporal weighted model on mapreduce for short-term traffic flow forecasting
Neurocomputing
(2016)
Z. Zeng et al.
Multi-target deep neural networks: theoretical analysis and implementation
Neurocomputing
(2018)
C. M. Bishop, Mixture density networks...
J. Bonneson et al.
Safety prediction methodology and analysis tool for freeways and interchanges
Natl. Cooperative Highway Res.
(2012)

J. Bruna, W. Zaremba, A. Szlam, Y. LeCun, Spectral networks and locally connected networks on graphs, arXiv:1312.6203...

X. Cao et al.

Interactive temporal recurrent convolution network for traffic prediction in data centers

IEEE Access

(2017)

H.-T.L. Chiang et al.

Safety, challenges, and performance of motion planners in dynamic environments

Robotics Research

(2020)

M. Defferrard et al.

Convolutional neural networks on graphs with fast localized spectral filtering

Advances in Neural Information Processing Systems

(2016)

K. El-Basyouny et al.

Comparison of two negative binomial regression techniques in developing accident prediction models

Transp. Res. Rec.

(2006)

R. Fu et al.

Using LSTM and GRU neural network methods for traffic flow prediction

2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC)

(2016)

A. Graves, Generating sequences with recurrent neural networks, arXiv:1308.0850...

C. Guarnaccia et al.

Short-term forecasting of tank water levels serving urban water distribution networks with Arima models

Frontiers in Water-Energy-NexusNature-Based Solutions, Advanced Technologies and Best Practices for Environmental Sustainability

(2020)

A. Hadayeghi et al.

Macrolevel accident prediction models for evaluating safety of urban transportation systems

Transp. Res. Rec.

(2003)

M. Henaff, J. Bruna, Y. LeCun, Deep convolutional networks on graph-structured data, arXiv:1506.05163...

T.N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, arXiv:1609.02907...

Cited by (34)

An attention-PCA based forecast combination approach to crude oil price
2024, Expert Systems with Applications
Crude oil price forecasting has garnered considerable attention due to its pivotal role in both market dynamics and economic stability. In this study, we present an attention-based principal component analysis (attention-PCA) methodology designed to improve the performance of oil price forecasting models. The attention-PCA approach enables greater focus on predictor variables with superior forecasting capabilities. Furthermore, we develop a diversity enhancement mechanism for forecast combination by incorporating multiple attention mechanisms, varying numbers of principal components, and a range of forecasting models. The empirical results demonstrates that attention-PCA-based individual forecasting models significantly outperform benchmark models, reducing the Mean Absolute Percentage Error (MAPE) by up to 43.2%. The proposed forecast combination strategy yields the most accurate and diverse forecasts among those evaluated, with the MAPE of the optimal combination model standing at 4.40%.
Perturbation-augmented Graph Convolutional Networks: A Graph Contrastive Learning architecture for effective node classification tasks
2024, Engineering Applications of Artificial Intelligence
In the context of recent advances in Graph Convolutional Networks (GCNs) for semi-supervised learning, a significant highlight is the potential of Graph Contrastive Learning (GCL). Many established GCL-based GCNs focus on developing various augmentation strategies and learning architectures to efficiently generate and learn self-supervised information. Unfortunately, the node representation fusion is often neglected in the process of learning representation. To address the issue, we propose a novel end-to-end GCL framework called Perturbation-augmented Graph Convolutional Networks (PA-GCN), by simultaneously considering supervised information and self-supervised information of graphs to fuse node representations from the input graph and the augmentation graphs. Essentially, PA-GCN develops a perturbation augmentation strategy based on original graph relations, serving as a basis to generate diverse node information and self-supervised information. Additionally, PA-GCN introduces three main components: A encoder for node representation learning; an attention mechanism for effective node representation fusion; and a discriminator capable of learning self-supervised information from unlabeled nodes, thereby enabling joint learning of supervised information, node representation fusion, and self-supervised information. We conduct experiments on five real world datasets to evaluate the performance of PA-GCN in semi-supervised node classification tasks. The experimental results demonstrate the strong adaptability of PA-GCN for base encoders and the superiority of PA-GCN over the most advanced methods.
MFAN: Mixing Feature Attention Network for trajectory prediction
2024, Pattern Recognition
Accurate trajectory prediction of surrounding agents is essential for autonomous vehicles, where the key challenge is to understand the complex interactions among agents. Previous works treat all interacted features between agents equally in modeling interaction, while neglecting their different importance to the interaction, thus inevitably limiting the interaction modeling ability. Besides, existing methods suffer from significant performance degradation when domain shifts, resulting in severely deviant prediction from reality. To address these issues, we propose a novel prediction framework, dubbed Mixing Feature Attention Network (MFAN). Specifically, the proposed mixing feature attention is a parallel design to adaptively determine the importance of different interacted features and simultaneously capture the global interaction feature to improve interaction modeling. Meanwhile, the spatial global interaction is modeled from a spatial edge-featured graph input to capture the enhanced spatial interaction. The temporal motion pattern is modeled from a temporal edge-featured graph input to enhance the domain adaption. Finally, we estimate the parameters of bivariant Gaussian distribution for trajectory prediction. Experimental results show that our method achieves superior performance in trajectory prediction while maintaining low computational complexity and performs accurate prediction even when domain shifts.
New machine learning application platform for spatial–temporal thermal error prediction and control with STFGCN for ball screw system
2023, Mechanical Systems and Signal Processing
The big data platform, which has a high control accuracy and efficiency, is expected to realize the high-accuracy prediction and real-time control of the thermal error (TE) for the ball screw system. But the TE of a ball screw system has significant dynamic spatial–temporal behaviors. The TE model, which fully captures its dynamic spatial–temporal behaviors, has not been reported so far. Moreover, the embedding of the TE model into the big data platform is extremely difficult, and the deployment of the big data platform is time-consuming and relies on the expert knowledge, and the cloud computing used in the big data platform also faces a limited bandwidth pressure of the industrial Internet, leading to the failure of the real-time control. To overcome the above limitations, a novel machine learning application platform (MLAP) is designed based on the cloud-terminal architecture. Moreover, the spatial–temporal fusion graph convolutional network (STFGCN) is proposed for the first time to fully capture of the dynamic spatial–temporal behaviors, and the proposed STFGCN is embedded into the designed MLAP to expedite the training process of the STFGCN model and reduce the bandwidth pressure on the industrial Internet. The prediction performance and robustness of the proposed STFGCN model are compared with that of the traditional multiple linear regression (MLR), long short-term memory (LSTM), gated recurrent unit (GRU), convolutional neural network-long short-term memory (CNN-LSTM), temporal-graph convolutional network (T-GCN), hypergraph neural network (HGNN), spatial–temporal graph convolutional network (STGCN), and attention-based spatial–temporal graph convolutional network (ASTGCN). The results suggest that the evaluation metrics R² of the MLR, LSTM, GRU, CNN-LSTM, T-GCN, HGNN, STGCN, ASTGCN, and STFGCN are 0.8039, 0.9500, 0.9441, 0.9414, 0.9571, 0.9512, 0.9689, 0.9712, and 0.9812 to show the prediction performance, respectively. Moreover, the training rate is improved by adding the number of the virtual machine nodes on the MLAP. The reduction rate of the machining error is the ratio of the machining error reduction to the original machining error. The positioning error and its fluctuation range of the ball screw system are reduced effectively, and the machining errors with the implementation of the MLAP are decreased by more than 33% and 72% compared with that with the pitch error control and without the TE control, respectively.
KSTAGE: A knowledge-guided spatial-temporal attention graph learning network for crop yield prediction
2023, Information Sciences
Accurate and timely crop yield prediction is difficult to achieve due to the nonlinear and dynamic spatial–temporal correlations included during the crop growth process. The latest approaches utilize deep learning techniques to implicitly capture sequential dependencies but fail to consider prior knowledge. In this paper, a novel Knowledge-guided Spatial–Temporal Attention Graph Network (KSTAGE), which incorporates prior knowledge to model dynamic temporal relationships and geospatial topology structures, is proposed. The proposed KSTAGE first utilizes a 3D convolutional neural network (CNN) to embed the initial spectral feature. Following its development, a new Knowledge-guided Temporal Multi-head Attention Algorithm (KTMA), the core of KSTAGE, is applied to automatically generate temporal attention weights from a self-attention mechanism under the guidance of prior knowledge. Furthermore, a new strategy for aligning self-attention scores with the prior distribution is developed. Finally, a location-aware Spatial Attention Graph Network utilizing geospatial knowledge is proposed to aggregate the spatial neighborhood features for the final yield prediction. The experimental results obtained on county-level yield prediction tasks in China and the contiguous United States (CONUS) demonstrate that the proposed KSTAGE achieves obvious improvements over the baselines.
Multi-point short-term prediction of station passenger flow based on temporal multi-graph convolutional network
2022, Physica A: Statistical Mechanics and its Applications
Prediction of passenger flow distribution in urban rail transit stations can provide important data support for passenger flow organization and passenger travel guidance. However, complex station space structure and simulation-based passenger flow data bring challenges to accurate analysis and prediction of the passenger flow inside the station. This paper proposes a temporal graph attention convolutional neural network model (TGACN) to predict the passenger flow volume and density in key areas of the station. Firstly, considering the topological structure of key areas and the characteristics of passenger flow and flow trend in the station, a multi-graph generation method for continuous space in stations is designed, including geographic neighborhood graph and semantic neighborhood graph, to represent the static and dynamic correlation between nodes. Secondly, a new method of spatio-temporal feature fusion is proposed, which takes multi-graph as input to optimize the extraction and expression of spatial and temporal correlation. Finally, the TGACN is verified by passenger flow data set, which is constructed based on real-time video monitoring data of a transit station in Guangzhou. Experiments demonstrate that the TGACN can obtain the spatio-temporal correlation from passenger flow data, and the prediction results are better than the existing baseline models.

View all citing articles on Scopus

View full text

STMAG: A spatial-temporal mixed attention graph-based convolution model for multi-data flow safety prediction

Highlights

Abstract

Introduction

Section snippets

Related work

Preliminaries

A case study on traffic safety prediction based on the STMAG model

Experiment

Conclusions

Funding

Author contributions

Declaration of Competing Interest

Acknowledgments

J. Saf. Res.

Appl. Comput. Harmon. Anal.

Neurocomputing

Transp. Res. Part C

Comput. Commun.

Inf. Sci.

Neurocomputing

Neurocomputing

Safety prediction methodology and analysis tool for freeways and interchanges

Natl. Cooperative Highway Res.

Interactive temporal recurrent convolution network for traffic prediction in data centers

IEEE Access

Safety, challenges, and performance of motion planners in dynamic environments

Robotics Research

Convolutional neural networks on graphs with fast localized spectral filtering

Advances in Neural Information Processing Systems

Comparison of two negative binomial regression techniques in developing accident prediction models

Transp. Res. Rec.

Using LSTM and GRU neural network methods for traffic flow prediction

2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC)

Short-term forecasting of tank water levels serving urban water distribution networks with Arima models

Frontiers in Water-Energy-NexusNature-Based Solutions, Advanced Technologies and Best Practices for Environmental Sustainability

Macrolevel accident prediction models for evaluating safety of urban transportation systems

Transp. Res. Rec.