Abstract
In this paper, we propose a novel approach which pursues cross-modal low-rank decomposition for robust multi-spectral foreground detection. For each spectrum, we employ the idea of low-rank and sparse decomposition to detect sparse moving objects against background with low-rank structure for its robustness to noises. Unlike simply combining multi-modal detecting results or compulsively enforcing a shared foreground mask in existing methods, we propose to pursue the cross modality consistency among heterogeneous modalities by introducing a soft cross-modality consistent constraint to the multi-modal low-rank decomposition model. Extensive experiments on the benchmark dataset GTFD suggest that our approach achieves superior performance over the state-of-the-art algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barnich, O., Droogenbroeck, M.V.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20, 1709–1724 (2011)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001)
Candes, E., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58, 175–181 (2011)
Chen, Y., Wang, J., Zhu, B., Tang, M., Lu, H.: Pixel-wise deep sequence learning for moving object detection. IEEE Trans. Circuits Syst. Video Technol., 1 (2017)
Cho, S.H., Kang, H.B.: Abnormal behavior detection using hybrid agents in crowded scenes. Pattern Recogn. Lett. 44, 64–70 (2014)
Davis, J., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106, 162–182 (2007)
Guo, X., Wang, X., Yang, L., Cao, X., Ma, Y.: Robust foreground detection using smoothness and arbitrariness constraints. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 535–550. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10584-0_35
Han, G., Cai, X., Wang, J.: Object detection based on combination of visible and thermal videos using a joint sample consensus background model. J. Softw. 8, 987–994 (2013)
Han, J., Bhanu, B.: Fusion of color and infrared video for moving human detection. Pattern Recogn. 40, 1771–1784 (2007)
Hong, R., Hu, Z., Wang, R., Wang, M., Tao, D.: Multi-view object retrieval via multi-scale topic models. IEEE Trans. Image Process. 25, 5814–5827 (2016)
Hong, R., Zhang, L., Zhang, C., Zimmermann, R.: Flickr circles: aesthetic tendency discovery by multi-view regularized topic modeling. IEEE Trans. Multimedia 18, 1555–1567 (2016)
Kolmogorov, V., Zabin, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 2, 147–159 (2004)
Li, C., Lin, L., Zuo, W., Wang, W., Tang, J.: An approach to streaming video segmentation with sub-optimal low-rank decomposition. IEEE Trans. Image Process. 25, 1947–1960 (2016)
Li, C., Wang, X., Zhang, L., Tang, J., Wu, H., Lin, L.: Weighted low-rank decomposition for robust grayscale-thermal foreground detection. IEEE Trans. Circuits Syst. Video Technol. 27, 725–738 (2017)
Li, C., Wu, X., Bao, Z., Tang, J.: ReGLe: spatially regularized graph learning for visual tracking. In: ACM, pp. 252–260 (2017)
Li, S.Z.: Markov Random Field Modeling in Image Analysis. Springer, London (2009). https://doi.org/10.1007/978-1-84800-279-1
Lim, L.A., Keles, H.Y.: Foreground segmentation using a triplet convolutional neural network for multiscale feature encoding. arXiv preprint arXiv:1801.02225 (2018)
Lucia, M., Alfredo, P.: A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans. Image Process. 17, 1168–1177 (2008)
Mazumder, R., Hastie, T., Tibshirani, R.: Spectral regularization algorithms for learning large incomplete matrices. J. Mach. Learn. Res. 11, 2287–2322 (2010)
Narayana, M., Hanson, A., Learned-miller, E.: Background modeling using adaptive pixelwise kernel variances in a hybrid feature space. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2012)
Oreifej, O., Li, X., Shah, M.: Simultaneous video stabilization and moving object detection in turbulence. IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013)
Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52, 471–501 (2010)
St-Charles, P.L., Bilodeau, G.A., Bergevin, R.: Subsense: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24, 359–373 (2014)
Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: 1999 Proceedings of the IEEE International Conference on Computer Vision, pp. 246–252 (1999)
Wang, X.: Intelligent multi-camera video surveillance: a review. Pattern Recogn. Lett. 34, 3–19 (2013)
Xin, B., Tian, Y., Wang, Y., Gao, W.: Background subtraction via generalized fused lasso foreground modeling. arXiv preprint, pp. 4676–4684 (2015)
Xu, M., Li, C., Shi, H., Tang, J., Zheng, A.: Moving object detection via integrating spatial compactness and appearance consistency in the low-rank representation. In: Yang, J., et al. (eds.) CCCV 2017. CCIS, vol. 773, pp. 50–60. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-7305-2_5
Yang, S., Luo, B., Li, C., Wang, G., Tang, J.: Fast grayscale-thermal foreground detection with collaborative low-rank decomposition. IEEE Trans. Circuits Syst. Video Technol., 1 (2017)
Ye, X., Yang, J., Sun, X., Li, K., Hou, C., Wang, Y.: Foreground-background separation from video clips via motion-assisted matrix restoration. IEEE Trans. Circuits Syst. Video Technol. 25, 1721–1734 (2015)
Zhang, H., Xu, D.: Fusing color and texture features for background model. In: Wang, L., Jiao, L., Shi, G., Li, X., Liu, J. (eds.) FSKD 2006. LNCS (LNAI), vol. 4223, pp. 887–893. Springer, Heidelberg (2006). https://doi.org/10.1007/11881599_110
Zhao, B., Li, Z., Liu, M., Cao, W., Liu, H.: Infrared and visible imagery fusion based on region saliency detection for 24-hours-surveillance systems. In: Proceeding of the IEEE International Conference on Robotics and Biomimetics (2013)
Zhou, X., Yang, C., Yu, W.: Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35, 597–610 (2013)
Acknowledgment
This work was partially supported by the National Natural Science Foundation of China (61502006, 61702002, 61472002 and 61671018) and the Natural Science Foundation of Anhui Higher Education Institutions of China (KJ2017A017).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, A., Zhao, Y., Li, C., Tang, J., Luo, B. (2018). Multispectral Foreground Detection via Robust Cross-Modal Low-Rank Decomposition. In: Hong, R., Cheng, WH., Yamasaki, T., Wang, M., Ngo, CW. (eds) Advances in Multimedia Information Processing – PCM 2018. PCM 2018. Lecture Notes in Computer Science(), vol 11164. Springer, Cham. https://doi.org/10.1007/978-3-030-00776-8_75
Download citation
DOI: https://doi.org/10.1007/978-3-030-00776-8_75
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00775-1
Online ISBN: 978-3-030-00776-8
eBook Packages: Computer ScienceComputer Science (R0)