Abstract
In this paper, we presented a fully integratedreal-time computer vision system that can detect and track multiple humans in a wide-area using a network of stereo cameras. Continuous human identities are achieved by fusing video tracking with different kinds of biometric devices. The system also provides immersive visualization which enables the users to conveniently navigate through space and time and query useful events. The key innovations include stereo-based multi-object detection and tracking, a unified approach for fusing multiple sensors of different modalities, visualization and user interface design.
Similar content being viewed by others
References
Haritaoglu, S., Harwood, D., Davis, L.S.: W4: Real-time surveillance of people and their activities. IEEE Trans. Pattern Recogn. Mach. Intell. 22(8) (2000)
Isard, M., MacCormick, J.: Bramble: a bayesian multiple-blob tracker. In: Proceedings of the International Conference On Computer Vision (2001)
Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. Washington DC, pp. II:406–413 (2004)
Ting, Yu., Ying, Wu.: Collaborative tracking of multiple targets. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. Washington DC, pp. I:834–841 (2004)
Tao H., Sawhney H.S., Kumar R. (2002) Object tracking with bayesian estimation of dynamic layer representations. IEEE Trans. Pattern Recogn. Mach. Intell. 24(1): 75–89
Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: Robust online appearance models for visual tracking. IEEE Trans. Pattern Recogn. Mach. Intell. 25(10) (2003)
Darrell T., Gordon G., Harville M., Woodfill J. (2000) Integrated person tracking using stereo, color, and pattern detection. Int. J. Comput. Vis. 37(2): 175–185
Beymer, D.: Person counting using stereo. In: Proceedings IEEE Workshop on Human Motion (2000)
Harville, M., Li, D.: Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2004)
Collins, R., Lipton, A., Fujiyoshi, H., Kanade, T.: Algorithms for cooperative multi-sensor surveillance. In: Proceedings of IEEE, 89, p. 1456 (2001)
Khan, S., Shah, M.: Consistent labeling of tracked objects in multiple cameras with overlapping fields of view. IEEE Trans. Pattern Recogn. Mach. Intell. 25(10) (2003)
Cupillard, F., Bremond, F., Thonnat, M.: Group behavior recognition with multiple cameras. In: Proceedings of the IEEE Workshop on Applications of Computer Vision (2002)
Pasula, H., Russell, S., Ostland, M., Ritov, Y.: Tracking many objects with many sensors. In: Proceedings of the International Joint Conference on Artificial Intelligence (1999)
Kettnaker, V., Zabih, R.: Bayesian multi-camera surveillance. In: Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (1999)
Javed, O., Rasheed, Z., Shafique, K., Shah, M.: Tracking across multiple cameras with disjoint views. In: Proceedings of the International Conference on Computer Vision (2003)
Haralick, R.M.: Propagating covariance in computer vision. In: Proceedings of the Workshop on Performance Characteristics of Vision Algorithms (1996)
Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Recogn. Mach. Intell. 24(5) (2002)
Mittal, A., Davis, L.S.: M2tracker: a multi-view approach to segmenting and tracking people in a cluttered scene. Int. J. Comput. Vis. 51(3) (2003)
Isard M., Blake A. (1998) Condensation-conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1): 5–28
Dempster A.P., Laird N.M., Rubin D.B. (1977) Maximum likelihood from incomplete data via the em algorithm. J R. Stat. Soc. B 39(1): 1–38
Kalman R. (1960) A new approach to linear filtering and prediction problems. J. Basic Eng. 82: 35–45
Goodwin and Sin: Adaptive Filtering, Prediction, and Control. Prentice Hall, Englewood Cliffs (1984)
Chris Stauffer.: Estimating tracking sources and sinks. In: Proceedings IEEE Workshop on Event Mining (2003)
LG Electronics USA, Inc.: LG IrisAccess(TM) 3000 system
Identix Inc.: FaceIt (R) ARGUS system
Duda O., Hart P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2000)
Sawnhey, H.S., Arpa, A., Kumar, R., Samarasekera, S., Aggarwal, M., Hsu, S.C., Nister, D., Hanna, K.J.: Video flashlights: real time rendering of multiple videos for immersive model visualization. In: Rendering Techniques, Pisa, Italy, pp. 157–168 (2002)
Multispectral Solutions, Inc.: Sapphire DART UWB-based Precision Asset Location System(TM)
Hartley, R., Zisserman A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhao, T., Aggarwal, M., Germano, T. et al. Toward a sentient environment: real-time wide area multiple human tracking with identities. Machine Vision and Applications 19, 301–314 (2008). https://doi.org/10.1007/s00138-008-0154-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-008-0154-y