Skip to main content
SpringerLink
Log in

Moving object detection using spatio-temporal multilayer compound Markov Random Field and histogram thresholding based change detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this article, we propose a Multi Layer Compound Markov Random Field (MLCMRF) Model to spatially segment different image frames of a given video sequence. The segmented image frames are combined with the change between the frames to detect the moving objects from a video. The proposed MLCMRF uses five Markov models in a single framework, one in spatial direction using color feature, four in temporal direction (using two color features and two edges/line fields). Hence, the proposed MLCMRF is a combination of spatial distribution of color, temporal color coherence and edge maps in the temporal frames. The use of such an edge preserving model helps in enhancing the object boundary in spatial segmentation and hence can detect moving objects with less effect of silhouette. A difference between the frames is used to generate the CDM and is subsequently updated with the previous frame video object plane (VOP) and the spatial segmentation of the consecutive frames, to detect the moving objects from the target image frames. Results of the proposed spatial segmentation approach are compared with those of the existing state-of-the-art techniques and are found to be better.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Abbreviations

(a,b):

Pixel position

t :

tth time instant

C D M t :

Change detection mask generated at time t

y :

Observed video sequence

y t :

Observed image frame at time t

T h :

Threshold

σ 2 :

Variance of each color plane

k :

Variance-covariance matrix

s :

Spatial site

p :

A neighboring site of s

y s t :

A site s of frame y t

x :

Segmentation of y

X t :

Markov random field at time t

x t :

Realization of X t , segmented version of y t

η s t :

Neighborhood of a site s in spatial direction of the tth frame

q :

A neighboring site of s

r :

A site in (t − 1) or (t − 2) th frame termed as temporal site

e :

A site of the Laplacian edge image x t , x t − 1 or x t − 2 called as edge site

V c (x t ):

Clique potential function in MRF

V s c (x s t , x q t ):

Clique potential function in the spatial domain

V t e c (x s t , x q r ):

Clique potential function in the temporal domain

V t e e c (x s t , x e r ):

Clique potential function incorporating edge features

α, β and γ :

MRF model bonding parameters

\( \hat {x}_{t} \) :

Estimated label or MAP estimate

𝜃 :

Parameter vector associated with x t

U(x t ):

Energy of realization x t

N :

Gaussian process

Z :

Partition function

f :

Number of features (for RGB space it is 3)

References

  1. Babacan SD, Pappas TN (2007) Spatiotemporal algorithm for background subtraction. In: Proceeding of IEEE international conference on acoustics, speech, and signal processing, pp 1065–1068

  2. Barnich O, Van Droogenbroeck M (2011) ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709–1724

    Article  MathSciNet  Google Scholar 

  3. Benedek C, Sziranyi T, Kato Z, Zerubia J (2009) Detection of object motion regions in aerial image pairs with a multilayer Markovian model. IEEE Trans Image Process 18(10):2303–2315

    Article  MathSciNet  Google Scholar 

  4. Besag J (1986) On the statistical analysis of dirty pictures. J R Stat Soc Ser B Methodol 48(3):259–302

    MathSciNet  MATH  Google Scholar 

  5. Bouwmans T (2012) Background subtraction for visual surveillance: a fuzzy approach. In: Pal S, Petrosino A, Maddalena L (eds) Handbook on soft computing for video surveillance, vol 5. Taylor and Francis Group, New York, pp 103–138

    Chapter  Google Scholar 

  6. Bovic AL (2000) Handbook of image and video processing. Academic Press, New York

    Google Scholar 

  7. Cho JH, Kim SD (2004) Object detection using spatio-temporal thresholding in image sequences. Electron Lett 40(18):1109–1110

    Article  Google Scholar 

  8. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  9. Deng Y, Manjunath BS (2001) Unsupervised segmentation of color-texture regions in images and video. IEEE Trans Pattern Anal Mach Intell 23(8):800–810

    Article  Google Scholar 

  10. Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90(7):1151–1163

    Article  Google Scholar 

  11. Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6(6):721–741

    Article  MATH  Google Scholar 

  12. Ghosh A, Mondal A, Ghosh S (2014) Moving object detection using markov random field and distributeddifferential evolution. Appl Soft Comput 15:121–135

    Article  Google Scholar 

  13. Gonzalez RC, Woods RE (2001) Digital image processing. Pearson Education, Singapore

    Google Scholar 

  14. Hinds RO, Pappas TN (1995) An adaptive clustering algorithm for segmentation of video sequences. In: Proceedings of international conference on acoustics, speech and signal processing, vol 4, pp 2427–2430

  15. Huang SS, Fu LC, Hsiao PY (2007) Region-level motion-based background modeling and subtraction using MRFs. IEEE Trans Image Process 16(5):1446–1456

    Article  MathSciNet  Google Scholar 

  16. Hwang SW, Kim EY, Park SH, Kim HJ (2001) Object extraction and tracking using genetic algorithms. In: Proceedings of international conference on image processing, vol 2, pp 383–386

  17. Kim EY, Park SH (2006) Automatic video segmentation using genetic algorithms. Pattern Recogn Lett 27(11):1252–1265

    Article  Google Scholar 

  18. Kirkpatrick SC, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  MathSciNet  MATH  Google Scholar 

  19. Lee DS, Yeom S, Son JY, Kim SH (2010) Automatic image segmentation for concealed object detection using the expectation-maximization algorithm. Opt Express 18(10):10659–10667

    Article  Google Scholar 

  20. Li SZ (2001) Markov random field modeling in image analysis. Springer, Japan

    Book  MATH  Google Scholar 

  21. Lin W, Sun MT, Poovendran R, Zhang Z (2008) Activity recognition using a combination of category components and local models for video surveillance. IEEE Trans Circuits Syst Video Technol 18(8):1128–1139

    Article  Google Scholar 

  22. Liu X, Qi C (2013) Future-data driven modeling of complex backgrounds using mixture of gaussians. Neurocomputing 119:439–453

    Article  Google Scholar 

  23. Liu X, Zhao G, Yao J, Qi C (2015) Background subtraction based on low-rank and structured sparse decomposition. IEEE Trans Image Process 24(8):2502–2514

    Article  MathSciNet  Google Scholar 

  24. Luthon F, Caplier A, Lievin M (1999) Spatiotemporal MRF approach to video segmentation: application to motion detection and lip segmentation. Signal Process 76 (1):61–80

    Article  MATH  Google Scholar 

  25. Oneata D, Revaud J, Verbeek J, Schmid C (2014) Spatio-temporal object detection proposals. In: European conference on computer vision, pp 737–752

  26. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66

    Article  Google Scholar 

  27. Palaniappan K, Ersoy I, Seetharaman G, Davis S, Kumar P, Rao RM, Linderman R (2011) Parallel flux tensor analysis for efficient moving object detection. In: Proceedings of the 14th international conference on information fusion, pp 535–546

  28. Qiao Y-L, Yuan K-L, Song C-Y, Xiang X-Z (2014) Detection of moving objects with fuzzy color coherence vector. Math Probl Eng 2014, Article ID 138065:8

  29. Rapantzikos K, Tsapatsoulis N, Avrithis Y, Kollias S (2009) Spatiotemporal saliency for video classification. Signal Process Image Commun 24(7):557–571

    Article  Google Scholar 

  30. Rougier C, Meunier J, St-Arnaud A, Rousseau J (2011) Robust video surveillance for fall detection based on human shape deformation. IEEE Trans Circuits Syst Video Technol 21(5):611–622

    Article  Google Scholar 

  31. Salember P, Marqués F (1999) Region based representation of image and video segmentation tools for multimedia services. IEEE Trans Circuits Syst Video Technol 9 (8):1147–1169

    Article  Google Scholar 

  32. Sasaki GH, Hajek B (1988) The time complexity of maximum matching by simulated annealing. J ACM 35(2):387–403

    Article  MathSciNet  MATH  Google Scholar 

  33. Satake J, Miura J (2009) Robust stereo-based person detection and tracking for a person following robot. In: Proceedings of IEEE international conference on robotics and automation, pp 1–6

  34. Sheikh Y, Shah M (2005) Bayesian modeling of dynamic scenes for object detection. IEEE Trans Pattern Anal Mach Intell 27(11):1778–1792

    Article  Google Scholar 

  35. Shi Q, Wang L, Cheng L, Smola A (2008) Discriminative human segmentation and recognition using semi-Markov model. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1–8

  36. Stauffer C, Grimson WEL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757

    Article  Google Scholar 

  37. Stolkin R, Greig A, Hodgetts M, Gilby J (2008) An EM/E-MRF algorithm for adaptive model based tracking in extremely poor visibility. Image Vis Comput 26 (4):480–495

    Article  Google Scholar 

  38. Su TF, Chen YL, Lai SH (2010) Over-segmentation based background modeling and foreground detection with shadow removal by using hierarchical MRFs. In: Proceedings of ACCV, pp 535–546

  39. Teixeira LF, Cardoso JS, Corte-Real L (2007) Object segmentation using background modelling and cascaded change detection. J Multimed 2(5):55–65

    Article  Google Scholar 

  40. Tekalp AM (1995) Digital video processing. Prentice Hall, New Jersey

    Google Scholar 

  41. Wang R, Bunyak F, Seetharaman G, Palaniappan K (2014) Static and moving object detection using flux tensor with split gaussian models. In: 2014 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 420–424

  42. Wu GK, Reed TR (1999) Image sequence processing using spatio temporal segmentation. IEEE Trans Circuits Syst Video Technol 9(5):798–807

    Article  Google Scholar 

  43. Xu D, Yan S, Tao D, Zhang L, Li X, Zhang HJ (2006) Human gait recognition with matrix representation. IEEE Trans Circuits Syst Video Technol 16(7):896–903

    Article  Google Scholar 

  44. Zhang YJ (2006) Advances in image and video segmentation. IRM Press, New York

    Book  Google Scholar 

  45. Zhou X, Yang C, Yu W (2013) Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(3):597–610

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology Goa, Ponda, Goa, 403401, India

    Badri Narayan Subudhi

  2. Department of Computer Science and Engineering, Jadavpur University, Kolkata, 700032, India

    Susmita Ghosh

  3. Department of Electronics and Telecommunication Engineering ITER, Siksha ‘O’, Anusandhan University, Bhubaneswar, 751030, India

    Pradipta Kumar Nanda

  4. Machine Intelligence Unit, Indian Statistical Institute, Kolkata, 700108, India

    Ashish Ghosh

Authors
  1. Badri Narayan Subudhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Susmita Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pradipta Kumar Nanda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ashish Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashish Ghosh.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Subudhi, B.N., Ghosh, S., Nanda, P.K. et al. Moving object detection using spatio-temporal multilayer compound Markov Random Field and histogram thresholding based change detection. Multimed Tools Appl 76, 13511–13543 (2017). https://doi.org/10.1007/s11042-016-3698-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3698-2

Keywords

Search

Navigation