Abstract
The inevitable problem of missing data is ubiquitous in the real transportation system, which makes the data-driven intelligent transportation system suffer from incorrect response. We propose a Bayesian robust Candecomp/Parafac (CP) tensor decomposition (BRCP) approach to deal with missing data and outliers by integrating the general form of transportation system domain knowledge. Specifically, when the lower rank tensor captures the global information, the sparse tensor is added to capture the local information, which can robustly predict the distribution of missing items and under the fully Bayesian treatment, the effective variational reasoning can prevent the over fitting problem. Real and reliable traffic data sets are used to evaluate the performance of the model in two data missing scenarios, which the experimental results show that the proposed BRCP model achieves the best imputation accuracy and is better than the most advanced baseline (Bayesian Gaussian CP decomposition (BGCP), high accuracy low-rank tensor completion (HaLRTC) and SVD-combined tensor decomposition (STD)), even in the case of high missed detection rate, the model still has the best performance and robustness.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ali A, Zhu Y, Zakarya M (2022) Exploiting dynamic Spatio-temporal graph convolutional neural networks for citywide traffic flows prediction[J]. Neural Netw 145:233–247
Bhanu M, Mendes-Moreira J, Chandra J (2020) Embedding traffic network characteristics using tensor for improved traffic prediction[J]. IEEE Trans Intell Transp Syst 99:1–13
Chen X, Sun L (2021) Bayesian temporal factorization for multidimensional time series prediction[J]. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2021.3066551
Chen X, He Z, Wang J (2018) Spatial-temporal traffic speed patterns discovery and incomplete data recovery via SVD-combined tensor decomposition[J]. Transportation Research Part C: Emerging Technologies 86:59–77
Chen X, Chen Y, He Z (2018) Urban traffic speed dataset of Guangzhou, China. https://doi.org/10.5281/zenodo.1205229.
Chen X, He Z, Sun L (2019) A Bayesian tensor decomposition approach for spatiotemporal traffic data imputation[J]. Transportation Research Part C Emerging Technologies 98:73–84
Chen X, He Z, Chen Y, et al (2019) Missing traffic data imputation and pattern discovery with a Bayesian augmented tensor factorization model[J]. Transportation research part C emerging technologies, 104(JUL.):66-77.
Gong C, Zhang Y (2020) Urban traffic data imputation with Detrending and tensor decomposition[J]. IEEE Access 8:11124–11137
Goulart JHDM, Kibangou AY, Favier G (2017) Traffic data imputation via tensor completion based on soft thresholding of Tucker Core[J]. Transportation Research Part C Emerging Technologies 85:348–362
Jia T, Yan P (2021) Predicting citywide road traffic flow using deep spatiotemporal neural networks[J]. IEEE Trans Intell Transp Syst 22(5):3101–3111
Kolda TG, Bader BW (2009) Tensor decompositions and applications[J]. SIAM Rev 51(3):455–500
Li Y, Chai S, Ma Z, Wang G (2021) A hybrid deep learning framework for long-term traffic flow prediction[J]. IEEE Access 9:11264–11271
Liu J, Musialski P, Wonka P, Ye J (2013) Tensor completion for estimating missing values in visual data[J]. IEEE Trans Pattern Anal Mach Intell 35(1):208–220
Ran B, Tan H, Wu Y, Jin PJ (2016) Tensor based missing traffic data completion with spatial-temporal correlation[J]. Physica A Statistical Mechanics & Its Applications 446:54–63
Rodrigues F, Henrickson K, Pereira FC (2019) Multi-output Gaussian processes for crowd sourced traffic data imputation[J]. IEEE Trans Intell Transp Syst 20(2):594–603
Tan H, Wu Y, Shen B, Jin PJ, Ran B (2016) Short-term traffic prediction based on dynamic tensor completion[J]. IEEE Trans Intell Transp Syst 17(8):2123–2133
Zhao Q, Zhang L, Cichocki A (2015) Bayesian CP factorization of incomplete tensors with automatic rank determination[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence 37(9):1751–1763
Zhao Q, Zhou G, Zhang L, Cichocki A, Amari SI (2016) Bayesian robust tensor factorization for incomplete multiway data[J]. IEEE Transactions on Neural Networks and Learning Systems 27(4):736–748
Acknowledgments
This work was financially supported by the National Natural Science Foundation of China (No.12071104).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhu, Y., Wang, W., Yu, G. et al. A Bayesian robust CP decomposition approach for missing traffic data imputation. Multimed Tools Appl 81, 33171–33184 (2022). https://doi.org/10.1007/s11042-022-13069-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-13069-7