Skip to main content

Advertisement

SpringerLink
Log in

Dual-attention network with multitask learning for multistep short-term speed prediction on expressways

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this study, a dual-attention network (DAN) with multitask learning is proposed to solve the short-term prediction problems of traffic speed. The proposed DAN includes a road-type attention module (RAM), which performs accurate short-term speed prediction using road-type attention scores, a low-speed attention module (LAM), which is trained on weighted samples and fits low speed, and a decision support module, which outputs either RAM or LAM by estimating the level of the predicted speed. DAN can improve the transfer in the feature and speed prediction task layers by learning-associated and time-dependent tasks. The Shanghai expressway dataset is used to test and compare the proposed method and 15 other techniques. The results show that DAN with a multitask loss function obtains the smallest mean squared error (MSE) and mean absolute percentage error (MAPE) in most cases. LAM efficiently improves the predictive accuracy of low-speed samples, whereas RAM performs better in terms of the overall error reduction. DAN achieves the largest R-squared of 0.93 with a small reduction in R-squared by 0.12% from the training data to the test data, thereby illustrating its excellent generalization. DAN outperforms the other models by at least 13.5% in terms of the MSE and by 5.07% in terms of the MAPE on different road types. Adding LAM effectively improves the MAPE by at least 21.4% over RAM without increasing the error of the other speed levels. In terms of the MSE, RAM outperforms DAN by 12.6% in the best case. This study proved that the short-term speed prediction based on DAN has the ability to improve the accuracy on low-speed level and the generalization on different road types.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Li T, Sun H, Wu J, Gao Z, Ge Y, Ding R (2019) Optimal urban expressway system in a transportation and land use interaction equilibrium framework. Transp A Transp Sci 15:1247

    Google Scholar 

  2. Yang Y, Li M, Yu J, He F (2020) Expressway bottleneck pattern identification using traffic big data—the case of ring roads in Beijing, China. J Intell Transp Syst 24:54

    Article  Google Scholar 

  3. Zheng K, Yao E, Zhang J, Zhang Y (2019) Traffic flow estimation on the expressway network using toll ticket data. IET Intel Transp Syst 13:886

    Article  Google Scholar 

  4. Gu Y, Lu W, Qin L, Li M, Shao Z (2019) Short-term prediction of lane-level traffic speeds: a fusion deep learning model. Transp Res Part C Emerg Technol 106:1

    Article  Google Scholar 

  5. Yang B, Sun S, Li J, Lin X, Tian Y (2019) Traffic flow prediction using LSTM with feature enhancement. Neurocomputing 332:320

    Article  Google Scholar 

  6. Chen Y, Chen C, Wu Q, Ma J, Zhang G, Milton J (2020) Spatial-temporal traffic congestion identification and correlation extraction using floating car data. J Intell Transp Syst 24:1

    Article  Google Scholar 

  7. Guo S, Lin Y, Li S, Chen Z, Wan H (2019) Deep spatial–temporal 3D convolutional neural networks for traffic data forecasting. IEEE Trans Intell Transp Syst 20:3913

    Article  Google Scholar 

  8. Yao H, Tang X, Wei H, Zheng G, Li Z (2019) Revisiting spatial-temporal similarity: a deep learning framework for traffic prediction. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, p 5668

  9. Moayedi H, Mosallanezhad M, Rashid ASA, Jusoh WAW, Muazu MA (2020) A systematic review and meta-analysis of artificial neural network application in geotechnical engineering: theory and applications. Neural Comput Appl 32:495

    Article  Google Scholar 

  10. Wang Y, Wang Q, Suo D, Wang T (2020) Intelligent traffic monitoring and traffic diagnosis analysis based on neural network algorithm. Neural Computing and Applications S. I: Intelligent Computing Methodologies in Machine learning for IoT Applications

  11. Yi H, Bui KN, Jung H (2019) Implementing a deep learning framework for short term traffic flow prediction. In: Proceedings of the 9th international conference on web intelligence, mining and semantics, vol. 1. Association for Computing Machinery, Seoul, South Korea

  12. Ting P, Wada T, Chiu Y, Sun M, Sakai K, Ku W, Jeng AA, Hwu J (2020) Freeway travel time prediction using deep hybrid model—taking Sun Yat-Sen freeway as an example. IEEE Trans Veh Technol 8:8257

    Article  Google Scholar 

  13. Chen M, Yu X, Liu Y (2018) PCNN: deep convolutional networks for short-term traffic congestion prediction. IEEE Trans Intell Transp Syst 19:3550

    Article  Google Scholar 

  14. Zhang S, Yao Y, Hu J, Zhao Y, Li S, Hu J (2019) Deep autoencoder neural networks for short-term traffic congestion prediction of transportation networks. Sensors 19:2229

    Article  Google Scholar 

  15. Avuglah RK, Adu-Poku KA, Harris E (2014) Application of ARIMA models to road traffic accident cases in Ghana. Int J Stat Appl 4:233

    Google Scholar 

  16. Williams BM, Durvasula PK, Brown DE (1998) Urban freeway traffic flow prediction: application of seasonal autoregressive integrated moving average and exponential smoothing models. Transp Res Rec 1644:132

    Article  Google Scholar 

  17. Chen J, Li K, Rong H, Bilal K, Li K, Philip SY (2019) A periodicity-based parallel time series prediction algorithm in cloud computing environments. Inf Sci 496:506

    Article  Google Scholar 

  18. Yu B, Song X, Guan F, Yang Z, Yao B (2016) k-Nearest neighbor model for multiple-time-step prediction of short-term traffic condition. J Transp Eng 142:4016018

    Article  Google Scholar 

  19. Cai P, Wang Y, Lu G, Chen P, Ding C, Sun J (2016) A spatio temporal correlative k-nearest neighbor model for short-term traffic multistep forecasting. Transp Res Part C Emerg Technol 62:21

    Article  Google Scholar 

  20. Luo C, Huang C, Cao J, Lu J, Huang W, Guo J, Wei Y (2019) Short-term traffic flow prediction based on least square support vector machine with hybrid optimization algorithm. Neural Process Lett 50:2305

    Article  Google Scholar 

  21. Zhu X, Fan Y, Zhang F, Ye X, Chen C, Yue H (2018) Multiple-factor based sparse urban travel time prediction. Appl Sci 8:279

    Article  Google Scholar 

  22. Alajali W, Zhou W, Wen S, Wang Y (2018) Intersection traffic prediction using decision tree models. Symmetry 10:386

    Article  Google Scholar 

  23. Lin W (2001) A Gaussian maximum likelihood formulation for short-term forecasting of traffic flow. In: 2001 IEEE intelligent transportation systems, vol 150. IEEE, Oakland, CA, USA

  24. Chen J, Li K, Bilal K, Li K, Philip SY (2018) A bi-layered parallel training architecture for large-scale convolutional neural networks. IEEE Trans Parallel Distrib Syst 30:965

    Article  Google Scholar 

  25. Chen J, Li K, Tang Z, Bilal K, Yu S, Weng C, Li K (2016) A parallel random forest algorithm for big data in a spark cloud computing environment. IEEE Trans Parallel Distrib Syst 28:919

    Article  Google Scholar 

  26. Lv Z, Xu J, Kai Z, Yin H, Zhou X (2018) LC-RNN: a deep learning model for traffic speed prediction. In: 27th International joint conference on artificial intelligence, vol 3470. AAAI, Stockholm, Sweden

  27. Fu R, Zhang Z, Li L (2016) Using LSTM and GRU neural network methods for traffic flow prediction. In: 2016 31st youth academic annual conference of Chinese association of automation, vol 324. IEEE, Wuhan, China

  28. Zheng Z, Chen W, Wu X, Chen PCY, Liu J (2017) LSTM network: a deep learning approach for short-term traffic forecast. IET Intell Transp Syst 11:68

    Article  Google Scholar 

  29. Zhang W, Yu Y, Qi Y, Shu F, Wang Y (2019) Short-term traffic flow prediction based on spatio-temporal analysis and CNN deep learning. Transp A Transp Sci 15:1688

    Google Scholar 

  30. Jia Y, Wu J, Du Y (2016) Traffic speed prediction using deep learning method. In: 2016 IEEE 19th international conference on intelligent transportation systems, vol. 1217. IEEE, Rio de Janeiro, Brazil

  31. Polson NG, Sokolov VO (2017) Deep learning for short-term traffic flow prediction. Transp Res Part C Emerg Technol 79:1

    Article  Google Scholar 

  32. Ma X, Dai Z, He Z, Ma J, Wang Y, Wang Y (2017) Learning traffic as images: a deep convolutional neural network for large-scale transportation network speed prediction. Sensors 17:818

    Article  Google Scholar 

  33. Do LN, Vu HL, Vo BQ, Liu Z, Phung D (2019) An effective spatial-temporal attention based neural network for traffic flow prediction. Transp Res Part C Emerg Technol 108:12

    Article  Google Scholar 

  34. Wang J, Qian G, Wu J, Liu G, Zhang X (2016) Traffic speed prediction and congestion source exploration: a deep learning method. In: 2016 IEEE 16th international conference on data mining, IEEE, Barcelona, Spain

  35. Cui Z, Ke R, Pu Z, Wang Y (2017) Deep bidirectional and unidirectional LSTM recurrent neural network for network-wide traffic speed prediction. In: International workshop on urban computing in conjunction with the ACM SIGKDD 2017, Association for Computing Machinery, Halifax, Canada

  36. Wang H, Xu J, Ma S (2018) Characteristic parameters model of traffic flow in ring expressway based on physical attributes. In: 15th scientific and technical conference “transport systems. theory and practice 2018”. Springer, Katowice, Poland

  37. Ao GC, Chen HW, Zhang HL (2017) Discrete analysis on the real traffic flow of urban expressways and traffic flow classification. Adv Transp Stud 1:23

    Google Scholar 

  38. Kan Z, Tang L, Kwan M, Ren C, Liu D, Li Q (2019) Traffic congestion analysis at the turn level using Taxis’ GPS trajectory data. Comput Environ Urban Syst 74:229

    Article  Google Scholar 

  39. Zhao J, Gao Y, Bai Z, Wang H, Lu S (2019) Traffic speed prediction under non-recurrent congestion: Based on LSTM method and BeiDou navigation satellite system data. IEEE Intell Transp Syst Mag 11:70

    Article  Google Scholar 

  40. Baxter J (2000) A model of inductive bias learning. J Artif Intell Res 12:149

    Article  MathSciNet  Google Scholar 

  41. Ciliberto C, Mroueh Y, Poggio T, Rosasco L (2015) Convex learning of multiple tasks and their structure. In: the 32nd international conference on machine learning, JMLR.org, Lille, France

  42. Ruder S (2017) An overview of multi-task learning in deep neural networks. arXiv:1706.05098

  43. Zhang K, Liu Z, Zheng L (2019) Short-term prediction of passenger demand in multi-zone level: temporal convolutional neural network with multi-task learning. IEEE Trans Intell Transp Syst 4:1480

    Google Scholar 

  44. Cheng S, Lu F, Peng P, Wu S (2019) Multi-task and multi-view learning based on particle swarm optimization for short-term traffic forecasting. Knowl Based Syst 180:116

    Article  Google Scholar 

  45. Mena-Yedra R, Casas J, Gavaldà R (2018) Assessing spatio temporal correlations from data for short-term traffic prediction using multi-task learning. Transp Res Procedia 34:155

    Article  Google Scholar 

  46. Zhang K, Zheng L, Liu Z, Jia N (2020) A deep learning based multitask model for network-wide traffic speed prediction. Neurocomputing 396:438

    Article  Google Scholar 

  47. Yang Z (2017) Analysis of traffic congestion based on shanghai road traffic state index. Traffic Transp 2:7

    Google Scholar 

  48. Tao Y, Zhang L, Zhang Y (2016) A projection-based decomposition for the scalability of evolvable hardware. Soft Comput 20:2205

    Article  Google Scholar 

  49. Zhao P, Wang X, Wu GEW (2019) Simulation-based dynamic traffic assignment modeling for urban expressway network: a case study of Suzhou expressway in China. In: 19th COTA international conference of transportation professionals, ASCE, Nanjing, China

  50. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Computer vision—ECCV 2014, Springer, Zurich, Switzerland

  51. Park J, Woo S, Lee J, Kweon IS (2020) A simple and light-weight attention module for convolutional neural networks. Int J Comput Vis 128:783

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant No. 61872259), the Natural Science Foundation of Jiangsu Province (Grant No. BK20160324) and the Natural Science Foundation of Jiangsu Colleges and Universities (Grant No. 16KJB580009)

Author information

Authors and Affiliations

  1. School of Rail Transportation, Soochow University, Suzhou, China

    Yanyun Tao, Guoqi Yue & Xiang Wang

Authors
  1. Yanyun Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guoqi Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YT designed, implemented the framework in programming, conducted the experiments and wrote the manuscript. XW contributed to design of the framework, collect and analysis the data, as well as review and editing of the manuscript. GY collected and processed the dataset of short-term traffic speed. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Xiang Wang.

Ethics declarations

Conflict of interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled,” dual-attention network with multitask learning for multi-step short-term speed prediction on Expressway”.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Tables 11, 12, 13, 14 and 15.

Table 11 MAPE/MSE(*10−3) 5-min on five road types and mixed type
Full size table
Table 12 MAPE/MSE (*10−2) 10-min on five road types and mixed type
Full size table
Table 13 MAPE/MSE (*10−2) 15-min on five road types and mixed type
Full size table
Table 14 MAPE/MSE obtained by RAM, LAM and DAN
Full size table
Table 15 Accuracy of DSM for 5-min prediction on different road types
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tao, Y., Yue, G. & Wang, X. Dual-attention network with multitask learning for multistep short-term speed prediction on expressways. Neural Comput & Applic 33, 7103–7124 (2021). https://doi.org/10.1007/s00521-020-05478-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05478-2

Keywords

Search

Navigation