skip to main content
10.1145/2510650.2510653acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Nobody likes Mondays: foreground detection and behavioral patterns analysis in complex urban scenes

Published: 21 October 2013 Publication History

Abstract

Streams of images from large numbers of surveillance webcams are available via the web. The continuous monitoring of activities at different locations provides a great opportunity for research on the use of vision systems for detecting actors, objects, and events, and for understanding patterns of activity and anomaly in real-world settings. In this work we show how images available on the web from surveillance webcams can be used as sensors in urban scenarios for monitoring and interpreting states of interest such as traffic intensity. We highlight the power of the cyclical aspect of the lives of people and of cities. We extract from long-term streams of images typical patterns of behavior and anomalous events and situations, based on considerations of day of the week and time of day. The analysis of typia and atypia required a robust method for background subtraction. For this purpose, we present a method based on sparse coding which outperforms state-of-the-art works on complex and crowded scenes.

References

[1]
A. Abrams, J. Tucek, J. Little, N. Jacobs, and R. Pless. Lost: Longterm Observation of Scenes (with Tracks). IEEE Workshop on Applications of Computer Vision (WACV), 2012.
[2]
L. Bo, X. Ren, and D. Fox. Unsupervised feature learning for rgb-d based object recognition. International Symposium on Experimental Robotics, (ISER), 2012.
[3]
T. Bouwmans. Recent advanced statistical background modeling for foreground detection: A systematic survey. Recent Patents on Computer Science, 4(3):147--176, 2011.
[4]
M. D. Breitenstein, H. Grabner, and L. V. Gool. Hunting nessie -- real-time abnormality detection from webcams. IEEE Int. Workshop on Visual Surveillance, 2009.
[5]
S. Brutzer, B. Hoferlin, and G. Heidemann. Evaluation of background subtraction techniques for video surveillance. CVPR, 2011.
[6]
E. J. Candes, X. Li, Y. Ma, and J. Wright. Robust principal component analysis' Journal of ACM, 58(1):1--37, 2009.
[7]
V. Cevher, C. Hegde, M. F. Duarte, and R. G. Baraniuk. Sparse signal recovery using markov random fields. NIPS, 2007.
[8]
V. Cevher, A. Sankaranarayanan, M. Duarte, D. Reddy, R. Baraniuk, and R. Chellappa. Compressive sensing for background subtraction. ECCV, 2008.
[9]
X. Cui, J. Huang, S. Zhang, and D. Metaxas. Background subtraction using group sparsity and low rank constraint. ECCV, 2012.
[10]
N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. CVPR, 2005.
[11]
M. Dikmen and T. S. Huang. Robust estimation of foreground in surveillance videos by sparse error estimation. ICPR, 2008.
[12]
I. J. Goodfellow, Q. V. Le, A. M. Sav.e, H. L, and A. Y. Ng. Measuring invariance in deep networks. NIPS, 2009.
[13]
B. Han and L. S. Davis. Density-based multifeature background subtraction with support vector machine. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34:1017--1023, 2012.
[14]
N. Jacobs, N. Roman, and R. Pless. Consistent temporal variations in many outdoor scenes. CVPR, 2007.
[15]
D. Kuettel, M. D. Breitenstein, L. V. Gool, and V. Ferrari. What's going on? Discovering spatio-temporal dependencies in dynamic scenes. CVPR, 2010.
[16]
J. Li, S. Gong, and T. Xiang. Global behaviour inference using probabilistic latent semantic analysis. BMVC, 2008.
[17]
J. Ngiam, C. Y. Foo, Y. Mai, C. Suen, and A. Ng. Unsupervised feature learning and deep learning tutorial. http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial.
[18]
R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML, 2007.
[19]
V. Reddy, C. Sanderson, and B. C. Lovell. Improved foreground detection via block-based classifier cascade with probabilistic decision integration. IEEE Transactions on Circuits and Systems for Video Technology, 23(1):83--93, 2013.
[20]
E. Ricci, G. Zen, N. Sebe, and S. Messelodi. A prototype learning framework using EMD: Application to complex scenes analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(3):513--526, 2013.
[21]
S. Rifai, Y. Bengio, A. Courville, P. Vincent, and M. Mirza. Disentangling factors of variation for facial expression recognition. ECCV, 2012.
[22]
S. Rifai, G. Mesnil, P. Vincent, X. Muller, Y. Bengio, Y. Dauphin, and X. Glorot. Higher order contractive auto-encoder. ECML, 2011.
[23]
R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. Convolutional-recursive deep learning for 3d object classification. NIPS, 2012.
[24]
C. Stauffer and W. E. L. Grimson. Adaptive background mixture models for real-time tracking. CVPR, 1999.
[25]
K. Toyama, J. Krumm, B. Brumitt, and B. Meyers. Wallflowers: Principles and practise of background maintainance. ICCV, 1999.
[26]
J. Yang, K. Yu, and T. Huang. Efficient highly over-complete sparse coding using a mixture model. ECCV, 2010.
[27]
J. Yao and J.-M. Odobez. Multi-layer background subtraction based on color and texture. CVPR Visual Surveillance workshop (CVPR-VS), 2007.
[28]
G. Yu, G. Sapiro, and S. Mallat. Solving inverse problems with piecewise linear estimators: from gaussian mixture models to structured sparsity. IEEE Transactions on Image Processing, 21(5):2481--2499, 2012.
[29]
M. Zeiler, G. Taylor, and R. Fergus. Adaptive deconvolutional networks for mid and high level feature learning. ICCV, 2011.
[30]
C. Zhao, X. Wang, and W. Kuen Cham. Background subtraction via robust dictionary learning. EURASIP J. Image and Video Processing, 2011.
[31]
Y. Zhao, H. Gong, Y. Jia, and S.-C. Zhu. Background modeling by subspace learning on spatio-temporal patches. Pattern Recognition Letters, 2012.
[32]
Z. Zivkovic. Improved adaptive gaussian mixture model for background subtraction. ICPR, 2004.

Cited By

View all
  • (2018)Scene shape estimation from multiple partly cloudy daysComputer Vision and Image Understanding10.1016/j.cviu.2014.10.002134:C(116-129)Online publication date: 31-Dec-2018
  • (2018)Lost in Time: Temporal Analytics for Long-Term Video SurveillanceComputational Science and Technology10.1007/978-981-10-8276-4_33(347-357)Online publication date: 24-Feb-2018
  • (2015)Lost and found: Identifying objects in long-term surveillance videos2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA)10.1109/ICSIPA.2015.7412171(99-104)Online publication date: Oct-2015
  • Show More Cited By

Index Terms

  1. Nobody likes Mondays: foreground detection and behavioral patterns analysis in complex urban scenes

      Recommendations

      Reviews

      Svetlana Segarceanu

      Image streams are often analyzed in order to monitor general activities and draw statistical conclusions about behavior. This paper proposes a method for inspecting image data by distinguishing the foreground elements from the background within a sequence of frames. Background modeling is based on a feature dictionary, where sparse features are obtained using a coding/decoding procedure to characterize local areas. The novelty and contribution of the approach reside in the fact that it works at the local patch level, providing weighted representatives. The foreground extraction is accomplished using an adaptive algorithm based on Gaussian mixture modeling, inspired by Zivkovic's work [1]. This method suits the nature of the test imagery material, which exhibits low frame rates and lighting conditions with a specific signal-to-noise ratio. Using a deviation measure based on the percentage of foreground pixels, the approach also spots inconsistent activities (such as the one that inspired the paper's title). The method evaluates the auto-encoder technique and compares it with other state-of-the-art work using a performance measure based on precision and recall values. The experiments aim to detect the daily patterns of behavior within a certain time interval based on a stream of webcam images of Fifth Avenue in New York City, collected by EarthCam Network2 over about four weeks in December 2011. Among the findings is the discovery that there is less traffic on Sunday nights, possibly indicating that, "even in New York City, the city that never sleeps, people seem to have more bed time before the beginning of new work weeks." The material is innovative, dense, interesting, and clearly explained, except for some minor errors. For example, I was unable to locate figure 3a. Online Computing Reviews Service

      Access critical reviews of Computing literature here

      Become a reviewer for Computing Reviews.

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      ARTEMIS '13: Proceedings of the 4th ACM/IEEE international workshop on Analysis and retrieval of tracked events and motion in imagery stream
      October 2013
      94 pages
      ISBN:9781450323932
      DOI:10.1145/2510650
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 21 October 2013

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. auto-encoders
      2. background subtraction
      3. long-term patterns analysis
      4. unsupervised feature learning
      5. video surveillance

      Qualifiers

      • Research-article

      Conference

      MM '13
      Sponsor:
      MM '13: ACM Multimedia Conference
      October 21, 2013
      Barcelona, Spain

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)1
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 23 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2018)Scene shape estimation from multiple partly cloudy daysComputer Vision and Image Understanding10.1016/j.cviu.2014.10.002134:C(116-129)Online publication date: 31-Dec-2018
      • (2018)Lost in Time: Temporal Analytics for Long-Term Video SurveillanceComputational Science and Technology10.1007/978-981-10-8276-4_33(347-357)Online publication date: 24-Feb-2018
      • (2015)Lost and found: Identifying objects in long-term surveillance videos2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA)10.1109/ICSIPA.2015.7412171(99-104)Online publication date: Oct-2015
      • (2015)Deep learning driven blockwise moving object detection with binary scene modelingNeurocomputing10.1016/j.neucom.2015.05.082168:C(454-463)Online publication date: 30-Nov-2015
      • (2014)Lost WorldProceedings of the 29th International Conference on Image and Vision Computing New Zealand10.1145/2683405.2683436(224-229)Online publication date: 19-Nov-2014
      • (2014)Democratizing the visualization of 500 million webcam images2014 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)10.1109/AIPR.2014.7041925(1-5)Online publication date: Oct-2014

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media