Skip to main content
SpringerLink
Log in

Joint fuzzy background and adaptive foreground model for moving target detection

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Moving target detection is one of the most basic tasks in computer vision. In conventional wisdom, the problem is solved by iterative optimization under either Matrix Decomposition (MD) or Matrix Factorization (MF) framework. MD utilizes foreground information to facilitate background recovery. MF uses noise-based weights to fine-tune the background. So both noise and foreground information contribute to the recovery of the background. To jointly exploit their advantages, inspired by two framework complementary characteristics, we propose to simultaneously exploit the advantages of these two optimizing approaches in a unified framework called Joint Matrix Decomposition and Factorization (JMDF). To improve background extraction, a fuzzy factorization is designed. The fuzzy membership of the background/foreground association is calculated during the factorization process to distinguish their contributions of both to background estimation. To describe the spatio-temporal continuity of foreground more accurately, we propose to incorporate the first order temporal difference into the group sparsity constraint adaptively. The temporal constraint is adjusted adaptively. Both foreground and the background are jointly estimated through an effective alternate optimization process, and the noise can be modeled with the specific probability distribution. The experimental results of vast real videos illustrate the effectiveness of our method. Compared with the current state-of-the-art technology, our method can usually form the clearer background and extract the more accurate foreground. Anti-noise experiments show the noise robustness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Garcia-Garcia B, Bouwmans T, Silva A J R. Background subtraction in real applications: challenges, current models and future directions. Computer Science Review, 2020, 35: 100204

    Article  MathSciNet  Google Scholar 

  2. Stauffer C, Grimson W E L. Adaptive background mixture models for real-time tracking. In: Proceedings of 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1999, 246–252

  3. Weinland D, Boyer E. Action recognition using exemplar-based embedding. In: Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. 2008, 1–7

  4. Lin Z, Zhang H. Low-Rank Models in Visual Analysis: Theories, Algorithms, and Applications. Sea Harbor Drive Orlando: Academic Press, 2017

    Google Scholar 

  5. Vaswani N, Bouwmans T, Javed S, Narayanamurthy P. Robust subspace learning: robust PCA, robust subspace tracking, and robust subspace recovery. IEEE Signal Processing Magazine, 2018, 35(4): 32–55

    Article  Google Scholar 

  6. Yong H, Meng D, Zuo W, Zhang L. Robust online matrix factorization for dynamic background subtraction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(7): 1726–1740

    Article  Google Scholar 

  7. Zivkovic Z. Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition. 2004, 28Sea Harbor Drive Orlando31

  8. Elgammal A, Harwood D, Davis L. Non-parametric model for background subtraction. In: Proceedings of the 6th European Conference on Computer Vision. 2000, 751Sea Harbor Drive Orlando767

  9. J Candès E J, Li X, Ma Y, Wright J. Robust principal component analysis? Journal of the ACM, 2011, 58(3): 11

    Article  MathSciNet  MATH  Google Scholar 

  10. Chen J, Yang J. Robust subspace segmentation via low-rank representation. IEEE Transactions on Cybernetics, 2014, 44(8): 1432–1445

    Article  Google Scholar 

  11. Liu X, Zhao G, Yao J, Qi C. Background subtraction based on low-rank and structured sparse decomposition. IEEE Transactions on Image Processing, 2015, 24(8): 2502–2514

    Article  MathSciNet  MATH  Google Scholar 

  12. Guo X, Lin Z. Low-rank matrix recovery via robust outlier estimation. IEEE Transactions on Image Processing, 2018, 27(11): 5316–5327

    Article  MathSciNet  MATH  Google Scholar 

  13. Javed S, Mahmood A, Al-Maadeed S, Bouwmans T, Jung S K. Moving object detection in complex scene using spatiotemporal structured-sparse RPCA. IEEE Transactions on Image Processing, 2019, 28(2): 1007–1022

    Article  MathSciNet  MATH  Google Scholar 

  14. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning, 2011, 3(1): 1–122

    Article  MATH  Google Scholar 

  15. Zhou X, Yang C, Yu W. Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(3): 597–610

    Article  Google Scholar 

  16. Cao X, Yang L, Guo X. Total variation regularized RPCA for irregularly moving object detection under dynamic background. IEEE Transactions on Cybernetics, 2016, 46(4): 1014–1027

    Article  Google Scholar 

  17. Xie H B, Li C, Xu R Y D, Mengersen K. Robust kernelized Bayesian matrix factorization for video background/foreground separation. In: Proceedings of the 5th International Conference on Machine Learning, Optimization, and Data Science. 2019, 484–495

  18. Meng D, De La Torre F. Robust matrix factorization with unknown noise. In: Proceedings of 2013 IEEE International Conference on Computer Vision. 2013, 1337–1344

  19. Cao X, Chen Y, Zhao Q, Meng D, Wang Y, Wang D, Xu Z. Low-rank matrix factorization under general mixture noise distributions. In: Proceedings of 2015 IEEE International Conference on Computer Vision. 2015, 1493–1501

  20. Chu Y, Wu X, Liu T, Liu J. A basis-background subtraction method using non-negative matrix factorization. In: Proceedings of SPIE 7546, Second International Conference on Digital Image Processing. 2010, 75461A

  21. Li L, Hu Q, Li X. Moving object detection in video via hierarchical modeling and alternating optimization. IEEE Transactions on Image Processing, 2019, 28(4): 2021–2036

    Article  MathSciNet  Google Scholar 

  22. Li L, Huang W, Gu I Y H, Tian Q. Statistical modeling of complex backgrounds for foreground object detection. IEEE Transactions on Image Processing, 2004, 13(11): 1459–1472

    Article  Google Scholar 

  23. Brutzer S, Höferlin B, Heidemann G. Evaluation of background subtraction techniques for video surveillance. In: Proceedings of CVPR 2011. 2011, 1937–1944

  24. Bouwmans T. Recent advanced statistical background modeling for foreground detection - a systematic survey. Recent Patents on Computer Science, 2011, 4(3): 147–176

    Google Scholar 

  25. Nebili W, Farou B, Seridi H. Using resources competition and memory cell development to select the best GMM for background subtraction. International Journal of Strategic Information Technology and Applications, 2019, 10(2): 21–43

    Article  Google Scholar 

  26. Chen M, Wei X, Yang Q, Li Q, Wang G, Yang M H. Spatiotemporal GMM for background subtraction with superpixel hierarchy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6): 1518–1525

    Article  Google Scholar 

  27. Elguebaly T, Bouguila N. Background subtraction using finite mixtures of asymmetric Gaussian distributions and shadow detection. Machine Vision and Applications, 2014, 25(5): 1145–1162

    Article  Google Scholar 

  28. Yang Y, Han D, Ding J, Yang Y. An improved ViBe for video moving object detection based on evidential reasoning. In: Proceedings of 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems. 2016, 26–31

  29. Ramírez-Alonso G, Chacón-Murguía M I. Auto-adaptive parallel SOM architecture with a modular analysis for dynamic object segmentation in videos. Neurocomputing, 2016, 175: 990–1000

    Article  Google Scholar 

  30. Wang H C, Lai Y C, Cheng W H, Cheng C Y, Hua K L. Background extraction based on joint gaussian conditional random fields. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 28(11): 3127–3140

    Article  Google Scholar 

  31. Xie Y, Gu S, Liu Y, Zuo W, Zhang W, Zhang L. Weighted schatten p-norm minimization for image denoising and background subtraction. IEEE Transactions on Image Processing, 2016, 25(10): 4842–4857

    Article  MathSciNet  MATH  Google Scholar 

  32. Hu W, Yang Y, Zhang W, Xie Y. Moving object detection using tensor-based low-rank and saliently fused-sparse decomposition. IEEE Transactions on Image Processing, 2017, 26(2): 724–737

    Article  MathSciNet  MATH  Google Scholar 

  33. Yin L, Parekh A, Selesnick I. Stable principal component pursuit via convex analysis. IEEE Transactions on Signal Processing, 2019, 67(10): 2595–2607

    Article  MathSciNet  MATH  Google Scholar 

  34. Feng J, Xu H, Yan S. Online robust PCA via stochastic optimization. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 404–412

  35. Guo K, Liu L, Xu X, Xu D, Tao D. GoDec+: fast and robust low-rank matrix decomposition based on maximum correntropy. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2323–2336

    Article  MathSciNet  Google Scholar 

  36. Cao H, Shang X, Wang Y, Song M, Chen S, Chang C I. GO decomposition (GoDec) approach to finding low rank and sparsity matrices for hyperspectral target detection. In: Proceedings of 2020 IEEE International Geoscience and Remote Sensing Symposium. 2020, 2807–2810

  37. Shakeri M, Zhang H. COROLA: a sequential solution to moving object detection using low-rank approximation. Computer Vision and Image Understanding, 2016, 146: 27–39

    Article  Google Scholar 

  38. Javed S, Oh S H, Sobral A, Bouwmans T, Jung S K. OR-PCA with MRF for robust foreground detection in highly dynamic backgrounds. In: Proceedings of the 12th Asian Conference on Computer Vision. 2014, 284–299

  39. Javed S, Oh S H, Bouwmans T, Jung S K. Robust background subtraction to global illumination changes via multiple features-based online robust principal components analysis with Markov random field. Journal of Electronic Imaging, 2015, 24(4): 043011

    Article  Google Scholar 

  40. Li C, Wang X, Zhang L, Tang J, Wu H, Lin L. Weighted low-rank decomposition for robust grayscale-thermal foreground detection. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(4): 725–738

    Google Scholar 

  41. Zhu L, Hao Y, Song Y. L1/2 norm and spatial continuity regularized low-rank approximation for moving object detection in dynamic background. IEEE Signal Processing Letters, 2018, 25(1): 15–19

    Article  Google Scholar 

  42. Xu Y, Wu Z, Chanussot J, Mura M D, Bertozzi A L, Wei Z. Low-rank decomposition and total variation regularization of hyperspectral video sequences. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(3): 1680–1694

    Article  Google Scholar 

  43. Zhang J, Jia X, Hu J. Error bounded foreground and background modeling for moving object detection in satellite videos. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4): 2659–2669

    Article  Google Scholar 

  44. Wu M, Sun Y, Hang R, Liu Q, Liu G. Multi-component group sparse RPCA model for motion object detection under complex dynamic background. Neurocomputing, 2018, 314: 120–131

    Article  Google Scholar 

  45. Liu X, Zhao G. Background subtraction using multi-channel fused lasso. Electronic Imaging, 2019, 2019(11): 269

    Google Scholar 

  46. Xin B, Tian Y, Wang Y, Gao W. Background subtraction via generalized fused lasso foreground modeling. In: Proceedings of 2015 IEEE conference on Computer Vision and Pattern Recognition. 2015, 4676–4684

  47. Javed S, Oh S H, Sobral A, Bouwmans T, Jung S K. Background subtraction via superpixel-based online matrix decomposition with structured foreground constraints. In: Proceedings of 2015 IEEE International Conference on Computer Vision Workshop. 2015, 930–938

  48. Ye X, Yang J, Sun X, Li K, Hou C, Wang Y. Foreground-background separation from video clips via motion-assisted matrix restoration. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(11): 1721–1734

    Article  Google Scholar 

  49. Cao X, Zhao Q, Meng D, Chen Y, Xu Z. Robust low-rank matrix factorization under general mixture noise distributions. IEEE Transactions on Image Processing, 2016, 25(10): 4677–4690

    Article  MathSciNet  MATH  Google Scholar 

  50. Liu Q, Li X. Efficient low-rank matrix factorization based on ℓ1,ε-norm for online background subtraction. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(7): 4900–4904

    Article  Google Scholar 

  51. Zhu Z, Meng Y, Kong D, Zhang X, Guo Y, Zhao Y. To see in the dark: N2DGAN for background modeling in nighttime scene. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(2): 492–502

    Article  Google Scholar 

  52. Patil P W, Murala S. MSFgNet: a novel compact end-to-end deep network for moving object detection. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(11): 4066–4077

    Article  Google Scholar 

  53. Zhao C, Basu A. Dynamic deep pixel distribution learning for background subtraction. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11): 4192–4206

    Article  Google Scholar 

  54. Hou B, Liu Y, Ling N. A super-fast deep network for moving object detection. In: Proceedings of 2020 IEEE International Symposium on Circuits and Systems. 2020, 1–5

  55. Patil P W, Murala S, Dhall A, Chaudhary S. MsEDNet: multi-scale deep saliency learning for moving object detection. In: Proceedings of 2018 IEEE International Conference on Systems, Man, and Cybernetics. 2018, 1670–1675

  56. Yan L F, Tu X Y. Background modeling based on chebyshev approximation. Journal of System Simulation, 2008, 20(4): 944–946, 1001

    MathSciNet  Google Scholar 

  57. Stagliano A, Noceti N, Verri A, Odone F. Online space-variant background modeling with sparse coding. IEEE Transactions on Image Processing, 2015, 24(8): 2415–2428

    Article  MathSciNet  MATH  Google Scholar 

  58. Messelodi S, Modena M C, Segata N, Zanin M. A Kalman filter based background updating algorithm robust to sharp illumination changes. In: Proceedings of the 13th International Conference on Image Analysis and Processing. 2005, 163–170

  59. Giraldo J H, Javed S, Sultana M, Jung S K, Bouwmans T. The emerging field of graph signal processing for moving object segmentation. In: Proceedings of the 27th International Workshop on Frontiers of Computer Vision. 2021, 31–45

  60. Pólya G. Isoperimetric Inequalities in Mathematical Physics. Princeton: Princeton University Press, 1951

    MATH  Google Scholar 

  61. Tekalp A M. Digital Video Processing. 2nd ed. Upper Saddle River: Prentice Hall Press, 2015

    Google Scholar 

  62. Li L, Wang Z, Hu Q, Dong Y. Adaptive nonconvex sparsity based background subtraction for intelligent video surveillance. IEEE Transactions on Industrial Informatics, 2021, 17(6): 4168–4178

    Article  Google Scholar 

  63. Javed S, Narayanamurthy P, Bouwmans T, Vaswani N. Robust PCA and robust subspace tracking: a comparative evaluation. In: Proceedings of 2018 IEEE Statistical Signal Processing Workshop. 2018, 836–840

  64. Bouwmans T, Aybat N S, Zahzah E H. Handbook of Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing. Boca Raton: CRC Press, 2016

    Book  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant No. 61902106) and in part by the Natural Science Foundation of Hebei Province (No. F2020202028).

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Hebei University of Technology, Tianjin, 300401, China

    Dawei Zhang, Peng Wang, Yongfeng Dong & Linhao Li

  2. Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, 26506-6109, USA

    Xin Li

Authors
  1. Dawei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongfeng Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Linhao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linhao Li.

Additional information

Dawei Zhang received the BS degree from Tiangong University, China in 2019. He is currently a Master Candidate of Hebei University of Technology, China. His research interests include computer vision and machine learning.

Peng Wang received the Bachelor of Engineering degree in Computer Science and Technology, the Master of Engineering degree in Computer Science and Technology, and the PhD degree in Microelectronics and Solid State Electronics from Hebei University, China in 2001, 2004, and 2011, respectively. Now, he is an Associate Professor with the School of Artificial Intelligent, Hebei University of Technology, China. His current research interests include computer application technology, industrial vision, remote sensing image processing.

Yongfeng Dong received the BS and MS degrees in computer science and technology and the PhD degree in electrical theory and new technology from the Hebei University of Technology, China in 2000, 2003, and 2008, respectively. Since 2016, he has been a Professor with the School of Artificial Intelligence, Hebei University of Technology, China. His research interests include intelligent information processing, big data technology, robot and intelligent control, and software engineering. He is a Senior Member of the China Computer Federation, a member of the Education Committee of the China Computer Federation, the Standing Director of the Hebei Computer Federation, and the Director of the Hebei Institute of Electronics.

Linhao Li received the BS degree in applied mathematics, the MS degree in computational mathematics, and the PhD degree in computer science from Tianjin University, China in 2012, 2014, and 2019, respectively. He is currently an Associate Professor with the School of Artificial Intelligent, Hebei University of Technology, China. His current research interests include background modeling and foreground detection, sparse signal recovery, Incremental learning and knowledge tracing.

Xin Li received the BS degree with highest honors in electronic engineering and information science from University of Science and Technology of China, China in 1996, and the PhD degree in electrical engineering from Princeton University, USA in 2000. He was a Member of Technical Staff with Sharp Laboratories of America, Camas, WA from Aug. 2000 to Dec. 2002. Since Jan. 2003, he has been a faculty member in Lane Department of Computer Science and Electrical Engineering. Dr. Li was elected a Fellow of IEEE in 2017 for his contributions to image interpolation, restoration and compression.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, D., Wang, P., Dong, Y. et al. Joint fuzzy background and adaptive foreground model for moving target detection. Front. Comput. Sci. 18, 182306 (2024). https://doi.org/10.1007/s11704-022-2099-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-022-2099-0

Keywords

Search

Navigation