Abstract
This paper describes a method for building visual “maps” from video data using quantized descriptions of motion. This enables unsupervised classification of scene regions based upon the motion patterns observed within them. Our aim is to recognise generic places using a qualitative representation of the spatial layout of regions with common motion patterns. Such places are characterised by the distribution of these motion patterns as opposed to static appearance patterns, and could include locations such as train platforms, bus stops, and park benches. Motion descriptions are obtained by tracking image features over a temporal window, and are then subjected to normalisation and thresholding to provide a quantized representation of that feature’s gross motion. Input video is quantized spatially into N ×N pixel blocks, and a histogram of the frequency of occurrence of each vector is then built for each of these small areas of scene. Within these we can therefore characterise the dominant patterns of motion, and then group our spatial regions based upon both proximity and local motion similarity to define areas or regions with particular motion characteristics. Moving up a level we then consider the relationship between the motion in adjacent spatial areas, and can characterise the dominant patterns of motion expected in a particular part of the scene over time. The current paper differs from previous work which has largely been based on the paths of moving agents, and therefore restricted to scenes in which such paths are identifiable. We demonstrate our method in three very different scenes: an indoor room scenario with multiple chairs and unpredictable unconstrained motion, an underground station featuring regions where motion is constrained (train tracks) and regions with complicated motion and difficult occlusion relationships (platform), and an outdoor scene with challenging camera motion and partially overlapping video streams.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Fernyhough, J.H., Cohn, A.G., Hogg, D.C.: Generation of semantic regions from image sequences. In: Proc. European Conference on Computer Vision (ECCV), Cambridge, UK, pp. 475–484 (1996)
Laptev, I.: On space-time interest points. Journal of Computer Vision 64(2/3), 107–123 (2005)
Johnson, N., Hogg, D.C.: Learning the distribution of object tractories for event recognition. Image and Vision Computing 14(8), 609–615 (1996)
Stauffer, C., Grimson, E.: Learning patterns of activity using real-time tracking. IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI) 22(8), 747–757 (2000)
Makris, D., Ellis, T.: Learning semantic scene models from observing activity in visual surveillance. IEEE Transactions on Systems, Man and Cybernetics 35(3), 397–408 (2005)
McKenna, S.J., Charif, H.N.: Summarising contextual activity and detecting unusual inactivity in a supportive home environment. Pattern Analysis and Applications 7(4), 386–401 (2004)
KaewTraKulPong, P., Bowden, R.: Probabilistic learning of salient patterns across spatially separated, uncalibrated views. In: Intelligent Distributed Surveillance Systems, pp. 36–40 (2004)
Xiang, T., Gong, S.: Beyond tracking: Modelling activity and understanding behaviour. International Journal of Computer Vision 67(1), 21–51 (2006)
Bicego, M., Cristiani, M., Murino, V.: Unsupervised scene analysis: a hidden Markov model approach. Computer Vision and Image Understanding (CVIU) 102, 22–41 (2006)
Efros, A.A., Berg, A.C., Mori, G., Malik, J.: Recognizing action at a distance. In: Proc. International Conference on Computer Vision (ICCV), Nice, France (2003)
Laptev, I., Pérez, P.: Retrieving actions in movies. In: Proc. International Conference on Computer Vision (ICCV) (2007)
Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: Proc. European Conference on Computer Vision (ECCV), pp. 428–441 (2006)
Gryn, J.M., Wildes, R.P., Tsotsos, J.: Detecting motion patterns via direction maps with application to surveillance. In: Workshop on Applications of Computer Vision, pp. 202–209 (2005)
Colombo, A., Leung, V., Orwell, J., Velastin, S.A.: Markov models of periodically varying backgrounds for change detection. In: Visual Information Engineering, London, UK (2007)
Shi, J., Tomasi, C.: Good features to track. In: Proc. Computer Vision and Pattern Recognition (CVPR), pp. 593–600 (1994)
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)
Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon (1991)
Home Office Scientific Development Branch U.i-LIDS: Imagery library for intelligent detection systems , http://scienceandresearch.homeoffice.gov.uk/hosdb/cctv-imaging-technology/video-based-detection-systems/i-lids/
Boykov, Y., Veksler, O., Zabih, R.: Efficient approximate energy minimization via graph cuts. IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI) 20(12), 1222–1239 (2001)
Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI) 26(2), 147–159 (2004)
Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI) 26(9), 1124–1137 (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dee, H.M., Fraile, R., Hogg, D.C., Cohn, A.G. (2008). Modelling Scenes Using the Activity within Them. In: Freksa, C., Newcombe, N.S., Gärdenfors, P., Wölfl, S. (eds) Spatial Cognition VI. Learning, Reasoning, and Talking about Space. Spatial Cognition 2008. Lecture Notes in Computer Science(), vol 5248. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87601-4_28
Download citation
DOI: https://doi.org/10.1007/978-3-540-87601-4_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87600-7
Online ISBN: 978-3-540-87601-4
eBook Packages: Computer ScienceComputer Science (R0)