Skip to main content

Advertisement

Springer Nature Link
Log in

SW-BiLSTM: a Spark-based weighted BiLSTM model for traffic flow forecasting

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Accurate traffic flow forecasting (TFF) is significant for mitigating traffic congestion. To address the existing issues of calculation and storage in dealing with big traffic flow data using the traditional centralized models on a single machine, this paper presents a Spark-based Weighted Bidirectional Long Short-Term M emory (SW-BiLSTM) model to improve the robustness and accuracy of TFF. Specifically, the resilient distributed dataset (RDD) and the Kalman filter (KF) are utilized to preprocess large-scale trajectory data (e.g., GPS trajectories of taxicabs). Next, a distributed SW-BiLSTM model on Spark is put forward to enhance the accuracy and efficiency of TFF, combined with the normal distribution for weighing the influence degree of the interaction between adjacent road segments and the time window for implementing the optimization of BiLSTM. Finally, the experimental results on an empirical study with the real-world taxi GPS trajectory data indicate that, compared with ARIMA, LR, GNB, CNN, GRU, SAEs, BP, LSTM, and WND-LSTM (LSTM with a time window and a normal distribution), the MAPE value of SW-BiLSTM is decreased by 65.62%, 17.78%, 87.29%, 69.10%, 3.52%, 21.09%, 59.66%, 42.86%, and 1.22%, respectively. In particular, SW-BiLSTM is superior to BiLSTM with 15.83% accuracy improvement on average.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Alghamdi T, Elgazzar K, Bayoumi M, Sharaf T, Shah S (2019) Forecasting traffic congestion using ARIMA modeling. In: 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC). IEEE, pp 1227–1232

  2. Belhadi A, Djenouri Y, Djenouri D, Lin JC-W (2020) A recurrent neural network for urban long-term traffic flow forecasting. Appl Intell 50 (10):3252–3265

    Article  Google Scholar 

  3. Cong Y, Wang J, Li X (2016) Traffic flow forecasting by a least squares support vector machine with a fruit fly optimization algorithm. Procedia Eng 137:59–68

    Article  Google Scholar 

  4. Habtemichael FG, Cetin M (2016) Short-term traffic flow rate forecasting based on identifying similar traffic patterns. Transp Res Part C-Emerging Technol 66:61–78

    Article  Google Scholar 

  5. Jia Y, Wu J, Xu M (2017) Traffic flow prediction with rainfall impact using a deep learning method. J Adv Transp 2017:1–10

    Article  Google Scholar 

  6. Ke X, Shi L, Guo W, Chen D (2018) Multi-dimensional traffic congestion detection based on fusion of visual features and convolutional neural network. IEEE Trans Intell Transp Syst 20(6):2157–2170

    Article  Google Scholar 

  7. Kong F, Li J, Jiang B, Song H (2019) Short-term traffic flow prediction in smart multimedia system for internet of vehicles based on deep belief network. Futur Gener Comput Syst 93:460–472

    Article  Google Scholar 

  8. Li L, Qin L, Qu X, Zhang J, Wang Y, Ran B (2019) Day-ahead traffic flow forecasting based on a deep belief network optimized by the multi-objective particle swarm algorithm. Knowl-Based Syst 172:1–14

    Article  Google Scholar 

  9. Li M-W, Wang Y-T, Geng J, Hong W-C (2021) Chaos cloud quantum bat hybrid optimization algorithm. Nonlinear Dyn 103(1):1167–1193

    Article  Google Scholar 

  10. Li W, Niu Q, Zhang W, Pang J (2015) The application of spark in the power grid intelligent decision analysis platform. In: 2015 7th International Conference on Intelligent Human-Machine Systems and Cybernetics, vol 2. IEEE, pp 216–219

  11. Li Y, Chen D (2016) A learning-based comprehensive evaluation model for traffic data quality in intelligent transportation systems. Multimed Tools Appl 75(19):11683–11698

    Article  Google Scholar 

  12. Lin JC, Shao Y, Zhou Y, Pirouz M, Chen H (2019) A Bi-LSTM mention hypergraph model with encoding schema for mention extraction. Eng Appl Artif Intell 85:175–181

    Article  Google Scholar 

  13. Liu B, Cheng J, Cai K, Shi P, Tang X (2017) Singular point probability improve LSTM network performance for long-term traffic flow prediction. Natl Conf Theor Comput Sci:328–340

  14. Liu D, Jiang Y, Pei M, Liu S (2018) Emotional image color transfer via deep learning. Pattern Recogn Lett 110:16–22

    Article  Google Scholar 

  15. Liu G, Guo J (2019) Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 337:325–338

    Article  Google Scholar 

  16. Liu W, Shoji Y (2019) DeepVM: RNN-based vehicle mobility prediction to support intelligent vehicle applications. IEEE Trans Ind Inf 16(6):3997–4006

    Article  Google Scholar 

  17. Luo X, Niu L, Zhang S (2018) An algorithm for traffic flow prediction based on improved SARIMA and GA. KSCE J Civ Eng 22(10):4107–4115

    Article  Google Scholar 

  18. Lv Y, Duan Y, Kang W, Li Z, Wang F-Y (2014) Traffic flow prediction with big data: A deep learning approach. IEEE Trans Intell Transp Syst 16(2):865–873

    Google Scholar 

  19. Ma D, Sheng B, Jin S, Ma X, Gao P (2018) Short-term traffic flow forecasting by selecting appropriate predictions based on pattern matching. IEEE Access 6:75629–75638

    Article  Google Scholar 

  20. Mackenzie J, Roddick JF, Zito R (2018) An evaluation of HTM and LSTM for short-term arterial traffic flow prediction. IEEE Trans Intell Transp Syst 20(5):1847–1857

    Article  Google Scholar 

  21. Okutani I, Stephanedes YJ (1984) Dynamic prediction of traffic volume through Kalman filtering theory. Transp Res Part B-methodol 18(1):1–11

    Article  Google Scholar 

  22. Seo SB, Yadav P, Singh D (2020) LoRa based architecture for smart town traffic management system. Multimed Tools Appl:1–16

  23. Shahriari S, Ghasri M, Sisson SA, Rashidi T (2020) Ensemble of ARIMA: Combining parametric and bootstrapping technique for traffic flow prediction. Transportmetrica A: Transport Sci 16(3):1552–1573

    Article  Google Scholar 

  24. Shao H, Soong B-H (2016) Traffic flow prediction with long short-term memory networks (LSTMs). In: 2016 IEEE Region 10 Conference (TENCON). IEEE, pp 2986–2989

  25. Soua R, Koesdwiady A, Karray F (2016) Big-data-generated traffic flow prediction using deep learning and dempster-shafer theory. In: 2016 International joint conference on neural networks (IJCNN). IEEE, pp 3195–3202

  26. Tan M-C, Wong SC, Xu J-M, Guan Z-R, Zhang P (2009) An aggregation approach to short-term traffic flow prediction. IEEE Trans Intell Transp Syst 10(1):60–69

    Article  Google Scholar 

  27. Tang J, Chen X, Hu Z, Zong F, Han C, Li L (2019) Traffic flow prediction based on combination of support vector machine and data denoising schemes. Physica A-Stat Mech Appl 534:120642

    Article  Google Scholar 

  28. Tian Y, Zhang K, Li J, Lin X, Yang B (2018) LSTM-based traffic flow prediction with missing data. Neurocomputing 318:297–305

    Article  Google Scholar 

  29. Wang C, Ye Z (2016) Traffic flow forecasting based on a hybrid model. J Intell Transp Syst 20(5):428–437

    Article  Google Scholar 

  30. Wang J, Chen R, He Z (2019) Traffic speed prediction for urban transportation network: A path based deep learning approach. Transp Res Part C-Emerging Technol 100:372–385

    Article  Google Scholar 

  31. Wang S, Wang X, Wang S, Wang D (2019) Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting. Int J Electr Power Energy Syst 109:470–479

    Article  Google Scholar 

  32. Williams BM, Hoel LA (2003) Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: Theoretical basis and empirical results. J Transp Eng 129(6):664–672

    Article  Google Scholar 

  33. Wu Y, Tan H, Qin L, Ran B, Jiang Z (2018) A hybrid deep learning based traffic flow prediction method and its understanding. Transp Res Part C: Emerging Technol 90:166–180

    Article  Google Scholar 

  34. Xia D, Wang B, Li H, Li Y, Zhang Z (2016) A distributed spatial–temporal weighted model on mapreduce for short-term traffic flow forecasting. Neurocomputing 179:246–263

    Article  Google Scholar 

  35. Xia D, Yang N, Jiang S, Hu Y, Li Y, Li H, Wang L (2021) A parallel NAW-DBLSTM algorithm on Spark for traffic flow forecasting. Neural Comput Appl 34(2):1557–1575

    Article  Google Scholar 

  36. Xia D, Zhang M, Yan X, Bai Y, Zheng Y, Li Y, Li H (2021) A distributed WND-LSTM model on MapReduce for short-term traffic flow prediction. Neural Comput Appl 33(7):2393–2410

    Article  Google Scholar 

  37. Xiao J, Xiao Z, Wang D, Bai J, Havyarimana V, Zeng F (2019) Short-term traffic volume prediction by ensemble learning in concept drifting environments. Knowl-Based Syst 164:213–225

    Article  Google Scholar 

  38. Xu H, Jiang C (2020) Deep belief network-based support vector regression method for traffic flow forecasting. Neural Comput Appl 32(7):2027–2036

    Article  Google Scholar 

  39. Xu L, Wang H, Gulliver TA (2020) Outage probability performance analysis and prediction for mobile IoV networks based on ICS-BP neural network. IEEE Internet Things J 8(5):3524–3533

    Article  Google Scholar 

  40. Yang H, Dillon TS, Chen Y (2017) Optimized structure of the traffic flow forecasting model with a deep learning approach. IEEE Trans Neural Netw 28(10):2371–2381

    Article  Google Scholar 

  41. Yang H, Hu X (2016) Wavelet neural network with improved genetic algorithm for traffic flow time series prediction. Optik 127(19):8103–8110

    Article  Google Scholar 

  42. Yousfi S, Berrani S, Garcia C (2017) Contribution of recurrent connectionist language models in improving LSTM-based Arabic text recognition in videos. Pattern Recogn 64:245–254

    Article  Google Scholar 

  43. Yue C (2012) IPSO-BPNN for short-term traffic flow prediction. Comput Eng Appli 48(27):239–243

    Google Scholar 

  44. Zhang W, Yu Y, Qi Y, Shu F, Wang Y (2019) Short-term traffic flow prediction based on spatio-temporal analysis and CNN deep learning. Transportmetrica A: Transport Sci 15(2):1688–1711

    Article  Google Scholar 

  45. Zhang Z, Hong W-C (2021) Application of variational mode decomposition and chaotic grey wolf optimizer with support vector regression for forecasting electric loads. Knowl-Based Syst 228:107297

    Article  Google Scholar 

  46. Zhao L, Zhou Y, Lu H, Fujita H (2019) Parallel computing method of deep belief networks and its application to traffic flow prediction. Knowl-Based Syst 163:972–987

    Article  Google Scholar 

  47. Zhao Z, Chen W, Wu X, Chen PCY, Liu J (2017) LSTM network: A deep learning approach for short-term traffic forecast. IET Intell Transp Syst 11(2):68–75

    Article  Google Scholar 

  48. Zheng Z, Su D (2014) Short-term traffic volume forecasting: A k-nearest neighbor approach enhanced by constrained linearly sewing principle component algorithm. Transp Res Part C-Emerging Technol 43:143–157

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work described in this paper was supported in part by the National Natural Science Foundation of China (Grant nos. 61762020, 62162012, 61773321, and 62173278), the Science and Technology Talents Fund for Excellent Young of Guizhou (Grant no. QKHPTRC20195669), the Science and Technology Support Program of Guizhou (Grant no. QKHZC2021YB531), and the Scientific Research Platform Project of Guizhou Minzu University (Grant no.GZMUSYS[2021]04).

Author information

Authors and Affiliations

  1. College of Data Science and Information Engineering, Guizhou Minzu University, Guiyang, 550025, China

    Dawen Xia, Nan Yang & Shunying Jian

  2. Department of Automotive Engineering, Guizhou Traffic Technician and Transportation College, Guiyang, 550008, China

    Yang Hu

  3. College of Electronic and Information Engineering, Southwest University, Chongqing, 400715, China

    Huaqing Li

Authors
  1. Dawen Xia
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Nan Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Shunying Jian
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yang Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Huaqing Li
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Dawen Xia.

Ethics declarations

Conflict of Interests

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xia, D., Yang, N., Jian, S. et al. SW-BiLSTM: a Spark-based weighted BiLSTM model for traffic flow forecasting. Multimed Tools Appl 81, 23589–23614 (2022). https://doi.org/10.1007/s11042-022-12039-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12039-3

Keywords

Profiles

  1. Dawen Xia View author profile
  2. Huaqing Li View author profile