Skip to main content
SpringerLink
Log in

3D Target Scale Estimation and Target Feature Separation for Size Preserving Tracking in PTZ Video

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

To achieve size preserving tracking, in addition to controlling the camera’s pan and tilt motions to keep the object of interest in the camera’s field of view (FOV), the camera’s focal length is adjusted automatically to compensate for the changes in the target’s image size caused by the relative motion between the camera and the target. The estimation accuracy of these changes determines the effectiveness of the resulting zoom control. The existing method of choice for real-time target scale estimation applies structure from motion (SFM) based on the weak perspective projection model. In this paper we propose a target scale estimation algorithm with a linear solution based on the more advanced paraperspective projection model, which improves the accuracy of scale estimation by considering center offset. Another key issue in SFM based algorithms is the separation of target and background features, especially when composite camera (pan/tilt/zoom) and target motions are involved. This paper designs a fast target feature separation/grouping algorithm, the 3D affine shape method. The resulting separation automatically adapts to the target’s 3D geometry and motion and is able to accommodate a large amount of off-plane rotation, which most existing separation/grouping algorithms find difficult to achieve. Experimental results illustrate the effectiveness of the proposed scale estimation and feature separation algorithms in tracking translating and rotating objects with a PTZ camera while preserving their sizes. In comparison with the leading size preserving tracking algorithm described by Tordoff and Murray, our algorithm is able to reduce the cumulative tracking error significantly from 17.4% to 3.3%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bouguet, J. Y. (2003). Pyramidal implementation of the Lucas Kanade feature tracker. Intel Corporation, Microprocessor Research Labs.

  • Collins, R. T. (2003). Mean-shift blob tracking through scale space. In IEEE int’l conf. on computer vision and pattern recognition (pp. 234–240). Madison, WI, June 2003.

  • Collins, R. T. (2003). Mean-shift blob tracking through scale space. In IEEE int’l conf. on computer vision and pattern recognition (pp. 234–240). Madison, WI, Jun. 2003.

  • Costeira, J. P., & Kanade, T. (1998). A multibody factorization method for independently moving objects. Int’l Journal of Computer Vision, 29(3), 159–179.

    Article  Google Scholar 

  • Crossmann, E., & Victor, J. (2000). A closed-form solution for paraperspective reconstruction. In Int’l conf. on pattern recognition (pp. 1864–1867). Barcelona, Spain, Sept. 2000.

  • Denzler, J., Zobel, M., & Niemann, H. (2003). Information theoretic focal length selection for real-time active 3-D object tracking. In IEEE int’l conf. on computer vision (pp. 400–407). Nice, France, Oct. 2003.

  • Elgammal, A., Harwood, D., & Davis, L. (2000). Non-parametric model for background subtraction. In European conf. on computer vision (pp. 751–767). Dublin, Ireland, Jun. 2000.

  • Fayman, J. F., Sudarsky, O., & Rivlin, E. (1998). Zoom tracking. In IEEE int’l conf. on robotics and automation (pp. 1051–1059). Leuven, Belgium, May 1998.

  • Fayman, J. F., Sudarsky, O., Rivlin, E., & Rudzsky, M. (2001). Zoom tracking and its applications. Machine Vision and Applications, 13(1), 25–37.

    Article  Google Scholar 

  • Hager, G. D., & Belhumeur, P. N. (2003). Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10), 1025–1039.

    Article  Google Scholar 

  • Hatano, K., & Hashimoto, K. (2003). Image-based visual servo using zoom mechanism. In SICE annual conference (pp. 2443–2446). Fukui, Japan, Aug. 2003.

  • Hoad, P., & Illingworth, J. (1995). Automatic control of camera pan, zoom and focus for improving object recognition. In IEE int’l conf. on image processing and its applications (pp. 291–295, no. 401). Jul. 1995.

  • Issacson, E., & Keller, H. B. (1994). Analysis of numerical methods. New York: Dover.

    Google Scholar 

  • Kang, S. (2004). Fusion of color and shape for object tracking using PTZ cameras (Technical report). University of Tennessee, Knoxville.

  • Kehtarnavaz, N., & Oh, H. (2003). Development and real-time implementation of a rule-based auto-focus algorithm. Journal of Real-Time Image, 9, 197–203.

    Article  Google Scholar 

  • Kim, Y. O., Paik, J., Heo, J., Koschan, A., Abidi, B., & Abidi, M. (2003). Automatic face region tracking for highly accurate face recognition in unconstrained environments. In IEEE int’l conf. on advanced video and signal based surveillance (pp. 29–36). Miami, FL, Jul. 2003.

  • Krueger, V., Happe, A., & Sommer, G. (1999). Affine real-time face tracking using wavelets. In IEEE int’l conf. on computer vision (pp. 141–148). Corfu, Greece, Sept. 1999.

  • Krueger, V., & Sommer, G. (2000). Affine real-time face tracking using Gabor wavelet networks. In Int’l conf. on pattern recognition (pp. 127–130). Barcelona, Spain, Sept. 2000.

  • Kuo, T. K., Fu, L. C., Jean, J. H., Chen, P. Y., & Chan, Y. M. (2002). Zoom-based head tracker in complex environment. In IEEE int’l conf. on control applications (pp. 725–730). Giasgow, UK, Sept. 2002.

  • Lee, D., Kwon, O., Shin, J., & Paik, J. (2005). Optical flow-based sub-image selection for robust video stabilization. In Int’l. technical conf. on circuits/systems, computers and communications (Vol. 4, p. 1605). Jeju, Korea, Jul. 2005.

  • Lee, D., Yao, Y., Paik, J., Abidi, B., & Abidi, M. (2007). Dynamic video tracking using a PTZ camera based on background stabilization and histogram matching (Technical report). Image Processing and Intelligent Systems lab, Chung-Ang University, Feb. 2007.

  • Lindeberg, T. (1998). Feature detection with automatic scale selection. Int’l Journal on Computer Vision, 30(2), 77–116.

    Google Scholar 

  • Liptak, B. (1995). Instrument engineer’s handbook: process control (3rd ed.). Butterworth-Heinemann.

  • Matsuyama, T., Hiura, S., Wada, T., Murase, K., & Yoshioka, A. (2006). Dynamic memory: architecture for real time integration of visual perception, camera action, and network communication. In IEEE conf. on computer vision and pattern recognition (pp. 728–735). Jun. 2006.

  • Mirmehdi, M., Palmer, P. L., & Kittler, J. (1997). Towards optimal zoom for automatic target recognition. In Scandinavian conf. on image analysis (pp. 447–453). Lappeenranta, Finland, Jun. 1997.

  • Poelman, C. J., & Kanade, T. (1997). A paraperspective factorization method for shape and motion recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(3), 206–218.

    Article  Google Scholar 

  • Shah, H., & Morrell, D. (2004). An adaptive zoom algorithm for tracking targets using pan-tilt-zoom cameras. In IEEE int’l conf. on acoustics, speech, and signal processing (pp. 721–724). Montreal, Canada, May 2004.

  • Shaik, J. S., & Iftekharuddin, K. M. (2000). Detection and tracking of rotated and scaled targets using Hilbert wavelet transform (Technical report).

  • Shi, J., & Tomasi, C. (1994). Good features to track. In IEEE int’l conf. on computer vision and pattern recognition (pp. 593–600). Seattle, WA, Jun. 1994.

  • Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: a factorization method. Int’l Journal on Computer Vision, 9(2), 137–154.

    Article  Google Scholar 

  • Tordoff, B. J., & Murray, D. W. (2000). Reactive zoom control while tracking (Technical Report OUEL2228/00). Department of Engineering Science, Oxford University.

  • Tordoff, B. J., & Murray, D. W. (2001). Reactive zoom control while tracking using an affine camera. In British machine vision conference. Manchester, UK. Sept. 2001.

  • Tordoff, B. J., & Murray, D. W. (2003). Resolution vs. tracking error: zoom as a gain controller. In IEEE int’l conf. on computer vision and pattern recognition (pp. 273–280). Madison, WI, Jun. 2003.

  • Tordoff, B., & Murray, D. (2004). Reactive control of zoom while fixating using perspective and affine cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), 98–112.

    Article  Google Scholar 

  • Torresani, L., Yang, D., Alexander, G., & Bregler, C. (2001). Tracking and modeling non-rigid objects with rank constraints. In IEEE conf. on computer vision and pattern recognition (pp. 493–500). Kuauai, HI, Dec. 2001.

  • Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In IEEE int’l conf. on computer vision and pattern recognition (pp. 511–518). Kauai, HI, Dec. 2001.

  • Wei, J., & Li, Z. (2001). On active camera control and camera motion recovery with foveate wavelet transform. IEEE Transactions on Pattern Recognition and Machine Intelligence, 23(8), 896–903.

    Article  Google Scholar 

  • Woelk, F., Schiller, I., & Koch, R. (2005). An airborne Bayesian color tracking system. In IEEE intelligent vehicles symposium. Las Vegas, NV, Jun. 2005.

  • Yao, Y., Abidi, B., & Abidi, M. (2006a). Fusion of omnidirectional and PTZ cameras for accurate cooperative tracking. In IEEE int’l conf. on advanced video and signal based surveillance (pp. 46–51). Sydney, Australia, Nov. 2006.

  • Yao, Y., Abidi, B., & Abidi, M. (2006b). 3D Target scale estimation for size preserving in PTZ video tracking. In IEEE int’l conf. on image processing (pp. 2817–2820). Atlanta, GA, Oct. 2006.

  • Yao, Y., Abidi, B., Doggaz, N., & Abidi, M. (2006c). Evaluation of sharpness measures and search algorithms for the auto-focusing of high magnification images. In SPIE defense and security symposium. Orlando, FL, Apr. 2006.

Download references

Author information

Authors and Affiliations

  1. Imaging, Robotics, and Intelligent Systems Laboratory, Department of Electrical & Computer Engineering, University of Tennessee, Knoxville, TN, 37996, USA

    Yi Yao, Besma Abidi & Mongi Abidi

Authors
  1. Yi Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Besma Abidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mongi Abidi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Besma Abidi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yao, Y., Abidi, B. & Abidi, M. 3D Target Scale Estimation and Target Feature Separation for Size Preserving Tracking in PTZ Video. Int J Comput Vis 82, 244–263 (2009). https://doi.org/10.1007/s11263-008-0198-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-008-0198-5

Keywords

Search

Navigation