research-article

People tracking and segmentation using spatiotemporal shape constraints

Authors:

Yasushi Makihara,

Yasushi YagiAuthors Info & Claims

VNBA '08: Proceedings of the 1st ACM workshop on Vision networks for behavior analysis

Pages 31 - 38

https://doi.org/10.1145/1461893.1461900

Published: 31 October 2008 Publication History

Abstract

We present an efficient people tracking and segmentation algorithm for gait recognition. Even though most existing gait recognition algorithms assume that people have been tracked and that silhouettes are available for gait classification, tracking and segmentation are very difficult especially for articulated objects such as human beings. We improve the performance of tracking and segmentation based on spatiotemporal shape constraints. First of all, we track people using an adaptive mean-shift tracker which produces initial results consisting of bounding boxes and foreground likelihood images. The initial results, generally speaking, are not accurate enough to be applied in gait recognition directly. We refine the results by matching with silhouette templates sequences in a batch mode to find the optimal silhouette-based gait paths corresponding to the input. Since the process is computationally expensive, we propose a novel efficient distance computation method to accelerate the spatiotemporal silhouette matching. The spatiotemporal shape priors are embedded into the Min-Cut algorithm to segment people out. Experiments on indoor and outdoor sequences demonstrate the effectiveness of the proposed approach.

References

[1]

A. Agarwal and B. Triggs. "Recovering 3D Human Pose from Monocular Images." IEEE Trans. Pattern Anal. Mach. Intell. 28(1):44--58 (2006)

Digital Library

[2]

G. Boregefors. "Distance Transformations in Digital Images", Computer Vision, Graphics and Image Processing Vol. 34(3), pp. 344--371, 1986.

Digital Library

[3]

Y. Boykov and M-P. Jolly. "Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in n-d images," in Proc. of Int'l Conf. on Computer Vision pp. 105--112, 2001.

[4]

Y. Boykov, V. Kolmogorov. "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision." IEEE Trans. Pattern Anal. Mach. Intell. 26(9):1124--1137, 2004.

Digital Library

[5]

T. Brox, B. Rosenhahn, J. Weickert. "Three-dimensional shape knowledge for joint image segmentation and pose estimation," in Proc. 27th DAGM pp. 109--116, 2005.

[6]

M. Bray, P. Kohli, P. H. S. Torr. "PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts," in ECCV II, pp. 642--655, 2006.

[7]

R. T. Collins and Y. Liu. "On-line selection of discriminative tracking features," IEEE Trans. Pattern im-Anal. Mach. Intell. Vol. 27(10), pp. 1631--1643, 2005.

Digital Library

[8]

D. Comaniciu, V. Ramesh, and P. Meer. "Kernel-based object tracking," IEEE Trans. Pattern Anal. Mach. Intell. Vol. 25(5), pp. 564--577, 2003.

Digital Library

[9]

A. Criminisi, G. Cross, A. Blake and V. Kolmogorov. "Bilayer segmentation of live video," in Proc. of IEEE Conf. on Computer Vision and Patten Recognition track-pp. 53--60, 2006.

Digital Library

[10]

A. Elgammal and C-S. Lee. "Inferring 3D body pose from silhouettes using activity manifold learning," in Proc. of IEEE Conf. on Computer Vision and Patten recognition pp. II-681-II-688, 2005.,

[11]

D. Freedman and T. Zhang. "Interactive graph cut based segmentation with shape priors," in Proc. of IEEE Conf. on Computer Vision and Patten Recognition pp. 755--762, 2004.

Digital Library

[12]

V. Kolmogorov, A. Criminisi, A. Blake, G. Cross, and C. Rother. "Bi-layer segmentation of binocular stereo video," in Proc. of IEEE Conf. on Computer Vision and Patten recognition pp. 407--414, 2005.

Digital Library

[13]

D. Serby, E. Koller-Meier, L. J. Van Gool. "Probabilistic object tracking using multiple features." In Proc. of Int'l Conf. on Pattern Recognition (2)pp. 184--187, 2004.

Digital Library

[14]

R. Li, M-H Yang, S. Sclaroff, and T-P. Tian. "Monocular tracking of 3D human motion with a coordinated mixture of factor analyzers", in Proc. of European Conf. on Computer Vision pp. 323--330, 2006.

[15]

M. Marszalek, C. Schmid. "Accurate Object Localization with Shape Masks." in Proc. of IEEE Conf. on Computer Vision and Patten Recognition pp. 1--8, 2007.

[16]

Y. Makihara and R. Sagawa and Y. Mukaigawa and T. Echigo and Y. Yagi. "Gait Recognition Using a View Transformation Model in the Frequency Domain," in Proc. of European Conf. on Computer Vision pp. 151--163, 2006.

[17]

L. Rabiner and B. Juang. Fundamentals of speech recognition Prentice Hall, 1993.

Digital Library

[18]

Y. Rathi, N. Vaswani, A. Tannenbaum, A. J. Yezzi. "Particle Filtering for Geometric Active Contours with Application to Tracking Moving and Deforming Objects". In Proc. of Computer Vision and Pattern Recognition II, pp. 2--9, 2005.

Digital Library

[19]

H. Sidenbladh, M. J. Black, D. J. Fleet. "Stochastic tracking of 3D human figures using 2D image motion," in Proc. of European Conf. on Computer Vision pp. 323--330, 2000.

Digital Library

[20]

C. Sminchisescu and A. Jepson. "Generative modeling for continuous non-linearly embedded visual inference," in Proc. of Int'l Conf. on Machine Learning pp. 702--718, 200

Digital Library

[21]

M. Swain and D. Ballard, "Color Indexing," Int'l Journal of Computer Vision vol. 7, pp. 11--32, 1991.

Digital Library

[22]

K. R. Sloan Jr. and S. L. Tanimoto, "Progressive Refinement of Raster Images", IEEE Trans. on Computers Vol. 28(11), pp. 871--874, 1979.

Digital Library

[23]

K. Toyama and A. Blake. "Probabilistic tracking in a metric space," in Proc. of Int'l Conf. on Computer Vision pp. 50--57, 2001.

[24]

R. Urtasun, D. J. Fleet and P. Fua. "3D people tracking with Gaussian Process Dynamical Models", in Proc. of Conf. on Computer Vision and Pattern Recognition pp. 238--245, 2006.

Digital Library

[25]

A. Veeraraghavan, A. K. Roy-Chowdhury and R. Chellappa. "Matching shape sequences in video with applications in human movement analysis," IEEE Trans. Pattern Anal. Mach. Intell. Vol. 27(12), pp. 1896--1909, 2005.

Digital Library

[26]

J. Wang, Y. Yagi, "Integrating Color and Shape-texture Features for Adaptive Real-time Tracking", IEEE Trans. on Image Processing vol. 17, no. 2, 2008.

[27]

L. Wang, T. Tan, W. Hu, H. Ning. "Automaticgait recognition based on statistical shape analysis." IEEE Trans. on Image Processing 12(9), pp. 1120--1131, 2003.

Digital Library

[28]

P. Yin, A. Criminisi, J. Winn, and I. Essa. "Tree-based classifiers for bilayer video segmentation", in Proc. of Conf. on Computer Vision and Pattern Recognition pp. 1--8, 2007.

Cited By

Makihara YOgi GYagi Y(2018)Geometrically Consistent Pedestrian Trajectory Extraction for Gait Recognition2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS)10.1109/BTAS.2018.8698559(1-11)Online publication date: 22-Oct-2018
https://dl.acm.org/doi/10.1109/BTAS.2018.8698559
Wang JYagi Y(2010)Tracking and segmentation using min-cut with consecutive shape priorsPaladyn, Journal of Behavioral Robotics10.2478/s13230-010-0008-y1:1(73-79)Online publication date: 31-Mar-2010
https://doi.org/10.2478/s13230-010-0008-y
Wang JYagi YMakihara Y(2009)People tracking and segmentation using efficient shape sequences matchingProceedings of the 9th Asian conference on Computer Vision - Volume Part II10.1007/978-3-642-12304-7_20(204-213)Online publication date: 23-Sep-2009
https://dl.acm.org/doi/10.1007/978-3-642-12304-7_20

Index Terms

People tracking and segmentation using spatiotemporal shape constraints
1. Computing methodologies
  1. Machine learning

Recommendations

Automatic scene calibration for detecting and tracking people using a single camera

Segmenting and tracking of people is an important aim in the video analysis with multitudinous applications. The scene calibration enables the system to process the input video in a different way depending on the camera position and the scene ...
People tracking in an environment with multiple depth cameras

The hand-gesture-triggered calibration can build the geometry in a region of interest.The proposed interleaving-based skeleton obtaining can track more people.The proposed pairwise trajectory matching scheme manages occlusion situations. This paper ...
K-means based segmentation for real-time zenithal people counting
ICIP'09: Proceedings of the 16th IEEE international conference on Image processing

The paper presents an efficient and reliable approach to automatic people segmentation, tracking and counting, designed for a system with an overhead mounted (zenithal) camera. Upon the initial block-wise background subtraction, k-means clustering is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VNBA '08: Proceedings of the 1st ACM workshop on Vision networks for behavior analysis

October 2008

116 pages

ISBN:9781605583136

DOI:10.1145/1461893

General Chairs:
Hamid Aghajan
Stanford University, USA
,
Andrea Prati
University of Modena and Reggio Emilia, Italy

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM08

Sponsor:

MM08: ACM Multimedia Conference 2008

October 31, 2008

British Columbia, Vancouver, Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
316
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Makihara YOgi GYagi Y(2018)Geometrically Consistent Pedestrian Trajectory Extraction for Gait Recognition2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS)10.1109/BTAS.2018.8698559(1-11)Online publication date: 22-Oct-2018
https://dl.acm.org/doi/10.1109/BTAS.2018.8698559
Wang JYagi Y(2010)Tracking and segmentation using min-cut with consecutive shape priorsPaladyn, Journal of Behavioral Robotics10.2478/s13230-010-0008-y1:1(73-79)Online publication date: 31-Mar-2010
https://doi.org/10.2478/s13230-010-0008-y
Wang JYagi YMakihara Y(2009)People tracking and segmentation using efficient shape sequences matchingProceedings of the 9th Asian conference on Computer Vision - Volume Part II10.1007/978-3-642-12304-7_20(204-213)Online publication date: 23-Sep-2009
https://dl.acm.org/doi/10.1007/978-3-642-12304-7_20

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten