Abstract
Background subtraction is an important and fundamental step in video analysis and is a challenging task due to the dynamic background, bad weather, complex moving behaviors and huge amount of data in real-life applications. To address this issue, we proposed a new decomposition model based on the tensor robust principal component analysis, which makes full use of the continuity of time and the correlation of space. The Kronecker-basis-representation based tensor sparsity was introduced into the proposed model to constrain the spatio-temporal correlation of the video background and to enhance the consideration of low-rank characteristics, thus effectively reducing the interference of the dynamic background. The \({l_{1,1,2}}\) norm was used to constrain the sparseness of the spatio-temporal structure of the foreground, which strengthens the spatio-temporal continuity and tube sparsity of the video foreground and improves the accuracy of moving objects extraction. Experiments demonstrate that, in most cases, the proposed algorithm achieves superior performance in terms of F-measure scores and visual effect of separating the foreground and background of the video compared with state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bouwmans, T., Sobral, A., & Javed, S. (2017). Decomposition into low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a large-scale dataset. Computer Science Review, 23, 1–71.
Bouwmans, T., & Zahzah, E. H. (2014). Robust PCA via principal component pursuit: A review for a comparative evaluation in video surveillance. Computer Vision and Image Understanding, 122, 22–34.
Candes, E. J., Wakin, M. B., & Boyd, S. P. (2008). Enhancing sparsity by reweighted \(l_1\) minimization. Journal of Fourier analysis and applications, 14(5–6), 877–905.
Cao, W., Wang, Y., & Sun, J. (2016). Total variation regularized tensor RPCA for background subtraction from compressive measurements. IEEE Transactions on Image Processing, 25(9), 4075–4090.
Chen, C., He, B., & Ye, Y. (2016). The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Mathematical Programming, 155(1–2), 57–79.
Chen, M., Wei, X., & Yang, Q. (2018). Spatiotemporal GMM for background subtraction with superpixel hierarchy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6), 1518–1525.
Cherapanamjeri, Y., Gupta, K., & Jain, P. (2017). Nearly optimal robust matrix completion. In The 34th international conference on machine learning (pp. 797–805). JMLR. org.
Elgammal, A., Harwood, D., & Davis, L. (2000) Non-parametric model for background subtraction. In European conference on computer vision (pp. 751–767). Springer, Berlin
Gillis, N., & Vavasis, S. A. (2018). On the complexity of robust PCA and \(l_{1}\)-norm low-rank matrix approximation. Mathematics of Operations Research, 43, 1074–1082.
Goldfarb, D., & Qin, Z. (2014). Robust low-rank tensor recovery: Models and algorithms. SIAM Journal on Matrix Analysis and Applications, 35(1), 225–253.
Gong, P., Zhang, C., & Lu, Z. (2013). A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. In 30th international conference on machine learning (pp. 37–45). IEEE.
Javed, S., Mahmood, A., & Al-Maadeed, S. (2018a). Moving object detection in complex scene using spatiotemporal structured-sparse RPCA. IEEE Transactions on Image Processing, 1–1, 1007–1022.
Javed, S., Mahmood, A., Bouwmans, T., & Jung, S. K. (2018b). Spatiotemporal low-rank modeling for complex scene background initialization. IEEE Transactions on Circuits and Systems for Video Technology, 28(6), 1315–1329.
Kumar, S., & Yadav, J. S. (2016). Segmentation of moving objects using background subtraction method in complex environments. Radio Engineering, 25(2), 399–408.
Lcandes, E. J., Li, X., & Ma, Y. (2011). Robust principal component analysis? Journal of the ACM (JACM), 58(3), 11.
Li, L., Huang, W., & Gu, I. Y. H. (2004). Statistical modeling of complex backgrounds for foreground object detection. IEEE Transactions on Image Processing, 13(11), 1459–1472.
Liu, G., Lin, Z., & Yan, S. (2013). Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 171–184.
Lu, C., Feng, J., & Chen, Y. (2018). Tensor robust principal component analysis with a new tensor nuclear norm. arXiv preprint arXiv:1804.03728.
Mahadevan, V., & Vasconcelos, N. (2010). Spatiotemporal saliency in dynamic scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), 171–177.
Rezaei, B., & Ostadabbas, S. (2017). Background subtraction via fast robust matrix completion. In 2017 IEEE international conference on computer vision (ICCV) (pp. 1871–1879). IEEE.
Rezaei, B., & Ostadabbas, S. (2018). Moving object detection through robust matrix completion augmented with objectness. IEEE Journal of Selected Topics in Signal Processing, 12(6), 1313–1323.
Wan, M., Gu, G., & Qian, W. (2018). Total variation regularization term-based low-rank and sparse matrix representation model for infrared moving target tracking. Remote Sensing, 10(4), 510.
Wang, S., Wang, Y., & Chen, Y. (2018). Robust PCA using matrix factorization for background/foreground separation. IEEE Access, 6, 18945–18953.
Wang, Y., Jodoin, P. M., & Porikli, F. (2014). CDnet 2014: An expanded change detection benchmark dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 387–394). IEEE Computer Society.
Wang, Y., Yin, W., & Zeng, J. (2019). Global convergence of ADMM in nonconvex nonsmooth optimization. Journal of Scientific Computing, 78, 29–63.
Xie, Q., Zhao, Q., & Meng, D. (2018). Kronecker-basis-representation based tensor sparsity and its applications to tensor recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(8), 1888–1902.
Xu, Y., Wu, Z., & Chanussot, J. (2018). Joint reconstruction and anomaly detection from compressive hyperspectral images using Mahalanobis distance-regularized tensor RPCA. IEEE Transactions on Geoscience and Remote Sensing, 56(5), 2919–2930.
Yang, H., Qu, S., & Javed, S. (2017). Real-time vehicle detection and counting in complex traffic scenes using background subtraction model with low-rank decomposition. IET Intelligent Transport Systems, 12(1), 75–85.
Yong, H., Meng, D., & Zuo, W. (2018). Robust online matrix factorization for dynamic background subtraction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(7), 1726–1740.
Yuan, M., & Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1), 49–67.
Zhang, Z., Ely, G., & Aeron, S. (2014). Novel methods for multilinear data completion and de-noising based on tensor-SVD. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3842–3849). IEEE Computer Society.
Acknowledgements
This project is partially supported by the National Natural Science Foundation of China (11961010, 61661017, 61967005, 61362021), Guangxi Natural Science Foundation (2018GXNSFAA138169, 2017GXNSFBA198212), Guangxi Colleges and Universities Key Laboratory project of Data Analysis and Computation (LDAC201704).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, L., Liu, J. & Wang, X. Background subtraction with Kronecker-basis-representation based tensor sparsity and \(l_{1,1,2}\) norm. Multidim Syst Sign Process 32, 77–90 (2021). https://doi.org/10.1007/s11045-020-00729-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-020-00729-w