Skip to main content
SpringerLink
Log in

Systems and algorithms for autonomous and scalable crowd surveillance using robotic PTZ cameras assisted by a wide-angle camera

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

We report an autonomous surveillance system with multiple pan-tilt-zoom (PTZ) cameras assisted by a fixed wide-angle camera. The wide-angle camera provides large but low resolution coverage and detects and tracks all moving objects in the scene. Based on the output of the wide-angle camera, the system generates spatiotemporal observation requests for each moving object, which are candidates for close-up views using PTZ cameras. Due to the fact that there are usually much more objects than the number of PTZ cameras, the system first assigns a subset of the requests/objects to each PTZ camera. The PTZ cameras then select the parameter settings that best satisfy the assigned competing requests to provide high resolution views of the moving objects. We propose an approximation algorithm to solve the request assignment and the camera parameter selection problems in real time. The effectiveness of the proposed system is validated in both simulation and physical experiment. In comparison with an existing work using simulation, it shows that in heavy traffic scenarios, our algorithm increases the number of observed objects by over 210%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alt, H., Arkin, E. M., Brönnimann, H., Erickson, J., Fekete, S. P., Knauer, C., Lenchner, J., Mitchell, J. S. B., & Whittlesey, K. (2006). Minimum-cost coverage of point sets by disks. In Symposium on computational geometry (pp. 449–458), June 2006.

  • Arkin, E. M., Barequet, G., & Mitchell, J. S. B. (2006). Algorithms for two-box covering. In Symposium on computational geometry (pp. 459–467), June 2006.

  • Bimbo, A. D., & Pernici, F. (2005). Distant targets identification as an on-line dynamic vehicle routing problem using an active-zooming camera. In Visual surveillance and performance evaluation of tracking and surveillance (pp. 97–104).

  • Bodor, R., Morlok, R., & Papanikolopoulos, N. (2004). Dual-camera system for multi-level activity recognition. In Proceedings IEEE/RSJ international conference on intelligent robots and systems (IROS 2004) (Vol. 1, pp. 643–648), 2 Oct. 2004.

  • Costello, C. J., Diehl, C. P., Banerjee, A., & Fisher, H. (2004). Scheduling an active camera to observe people. In VSSN’04: Proceedings of the ACM 2nd international workshop on video surveillance & sensor networks (pp. 39–45). New York: ACM.

    Chapter  Google Scholar 

  • Elgammal, A., Harwood, D., & Davis, L. (2000). Non-parametric model for background subtraction. In 6th European conference on computer vision (ECCV) (Vol. 2, pp. 751–761), Dublin, Ireland, June/July 2000.

  • Eppstein, D. (1997). Fast construction of planar two-centers. In Proc. 8th ACM-SIAM sympos. discrete algorithms (pp. 131–138), January 1997.

  • Fiore, L., Fehr, D., Bodor, R., Drenner, A., Somasundaram, G., & Papanikolopoulos, N. (2008). Multi-camera human activity monitoring. Journal of Intelligent and Robotic Systems, 52(1), 5–43.

    Article  Google Scholar 

  • Hampapur, A., Pankanti, S., Senior, A., Tian, Y.-L., Brown, L., & Bolle, R. (2003). Face cataloger: multi-scale imaging for relating identity to location. In Proceedings IEEE conference on advanced video and signal based surveillance (pp. 13–20), July 2003.

  • Lim, S.-N., Davis, L., & Elgammal, A. (2003). Scalable image-based multi-camera visual surveillance system. In Proceedings IEEE conference on advanced video and signal based surveillance (pp. 205–212), July 2003.

  • Lim, S.-N., Davis, L. S., & Mittal, A. (2006). Constructing task visibility intervals for video surveillance. Multimedia Systems, 12(3), 211–226.

    Article  Google Scholar 

  • Lim, S.-N., Davis, L., & Mittal, A. (2007). Task scheduling in large camera networks. In 8th Asian conference on computer vision, Tokyo, Japan (pp. 397–407).

  • Megiddo, N., & Supowit, K. (1984). On the complexity of some common geometric location problems. SIAM Journal on Computing, 13, 182–196.

    Article  MATH  MathSciNet  Google Scholar 

  • Qureshi, F., & Terzopoulos, D. (2008). Smart camera networks in virtual reality. Proceedings of the IEEE, 96(10), 1640–1656.

    Article  Google Scholar 

  • Sommerlade, E., & Reid, I. (2008). Information-theoretic active scene exploration. In IEEE conference on computer vision and pattern recognition (CVRP), Anchorage, Alaska, USA.

  • Song, D. (2009). Sharing a vision: systems and algorithms for collaboratively-teleoperated robotic cameras. Berlin: Springer.

    Google Scholar 

  • Song, D., & Goldberg, K. (2007). Approximate algorithms for a collaboratively controlled robotic camera. IEEE Transactions on Robotics, 23(5), 1061–1070.

    Article  Google Scholar 

  • Song, D., van der Stappen, A. F., & Goldberg, K. (2006). Exact algorithms for single frame selection on multi-axis satellites. IEEE Transactions on Automation Science and Engineering, 3(1), 16–28.

    Article  Google Scholar 

  • Song, D., Qin, N., & Goldberg, K. (2008). Systems, control models, and codec for collaborative observation of remote environments with an autonomous networked robotic camera. Autonomous Robots, 24(4), 435–449.

    Article  Google Scholar 

  • Xu, Y., & Song, D. (2009). Systems and algorithms for autonomously simultaneous observation of multiple objects using robotic ptz cameras assisted by a wide-angle camera. In International conference on intelligent robots and systems (IROS), St. Louis, USA, October 2009.

  • Xu, Y., Song, D., Yi, J., & van der Stappen, F. (2008). An approximation algorithm for the least overlapping p-frame problem with non-partial coverage for networked robotic cameras. In IEEE international conference on robotics and automation (ICRA), Pasadena, CA (pp. 1011–1016), May 2008.

  • Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: a survey. ACM Computing Surveys, 38(4), 1–45.

    Article  Google Scholar 

  • Zhang, C., Liu, Z., Zhang, Z., & Zhao, Q. (2008). Semantic saliency driven camera control for personal remote collaboration. In IEEE 10th workshop on multimedia signal processing (pp. 28–33), Oct. 2008.

  • Zhou, X., Collins, R. T., Kanade, T., & Metes, P. (2003). A master-slave system to acquire biometric imagery of humans at distance. In IWVS’03: First ACM SIGMM international workshop on video surveillance (pp. 113–120). New York: ACM.

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, Texas A&M University, College Station, TX, 77843, USA

    Yiliang Xu & Dezhen Song

Authors
  1. Yiliang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dezhen Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dezhen Song.

Additional information

This work was supported in part by the National Science Foundation under IIS-0643298 and MRI-0923203.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(WMV 1.43 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xu, Y., Song, D. Systems and algorithms for autonomous and scalable crowd surveillance using robotic PTZ cameras assisted by a wide-angle camera. Auton Robot 29, 53–66 (2010). https://doi.org/10.1007/s10514-010-9188-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-010-9188-x

Keywords

Search

Navigation