Skip to main content
SpringerLink
Log in

Multiframe Many–Many Point Correspondence for Vehicle Tracking in High Density Wide Area Aerial Videos

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper presents a novel framework for tracking thousands of vehicles in high resolution, low frame rate, multiple camera aerial videos. The proposed algorithm avoids the pitfalls of global minimization of data association costs and instead maintains multiple object-centric associations for each track. Representation of object state in terms of many to many data associations per track is proposed and multiple novel constraints are introduced to make the association problem tractable while allowing sharing of detections among tracks. Weighted hypothetical measurements are introduced to better handle occlusions, mis-detections and split or merged detections. A two-frame differencing method is presented which performs simultaneous moving object detection in both. Two novel contextual constraints of vehicle following model, and discouragement of track intersection and merging are also proposed. Extensive experiments on challenging, ground truthed data sets are performed to show the feasibility and superiority of the proposed approach. Results of quantitative comparison with existing approaches are presented, and the efficacy of newly introduced constraints is experimentally established. The proposed algorithm performs better and faster than global, 1–1 data association methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  • Ablavsky, V., Thangali, A., & Sclaroff, S. (2008). Layered graphical models for tracking partially-occluded objects. In CVPR, Anchorage.

  • Bazzani, L., Cristani, M., & Murino, V. (2010). Collaborative particle filters for group tracking. In ICIP, Hong Kong.

  • Berclaz, J., Fleuret, F., Turetken, E., & Fua, P. (2011). Multiple object tracking using k-shortest paths optimization. IEEE Transactions on Pattern Analysis and Machine, 33(9), 1806–1819.

    Article  Google Scholar 

  • Breitenstein, M., Reichlin, F., Leibe, B., Koller-Meier, E., & Gool, L. V. (2009). Robust tracking-by-detection using a detector confidence particle filter. In ICCV, Kyoto.

  • Cox, I., & Hingorani, S. (1996). An efficient implementation of Reid’s multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking. IEEE TPAMI, 18(2), 138–150.

    Article  Google Scholar 

  • Cox, I., Miller, M., Danchick, R., & Newnam, G. (1997). A comparison of two algorithms for determining ranked assignments with application to multitarget tracking and motion correspondence. IEEE Transaction of AES, 33(1), 295–301.

    Google Scholar 

  • Cucchiara, R., Piccardi, M., & Mello, P. (2000). Image analysis and rule-based reasoning for a traffic monitoring system. IEEE Transaction of ITS, 1(2), 119–130.

    Google Scholar 

  • Danchick, R., & Newnam, G. (2006). Reformulating Reid’s MHT method with generalised Murty k-best ranked linear assignment algorithm. IEE Proceedings Radar, Sonar and Navigation, 153(1), 22.

    Article  Google Scholar 

  • Durrant-Whyte, H. (1988). Sensor models and multisensor integration. International Journal of Robotics Research, 7(6), 97–113.

    Article  Google Scholar 

  • Edie, L. (1960). Car following and steady state theory for non-congested traffic. Operations Research, 9, 66–76.

    Article  MathSciNet  Google Scholar 

  • Gazis, D., Herman, R., & Potts, R. (1959). Car following theory of steady state traffic flow. Operations Research, 7, 499–505.

    Article  MathSciNet  Google Scholar 

  • Grabner, H., Leistner, C., & Bischof, H. (2008). Semi-supervised on-line boosting for robust tracking. In ECCV, Marseille.

  • Hinton, G. (1999). Products of experts. In Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN 99) (pp. 1–6).

  • Huang, K., Wang, L., Tan, T., & Maybank, S. (2008). A real-time object detecting and tracking system for outdoor night surveillance. Pattern Recognition, 41, 432–444.

    Article  MATH  Google Scholar 

  • Kang, J., Cohen, I., & Medioni, G. (2003). Soccer player tracking across uncalibrated camera streams. In ICCV: PETS Workshop, Nice.

  • Kikuchi, S., & Chakroborty, P. (1992). Car following model based on fuzzy inference system. Transportation Research Record, 1365, 82–91.

    Google Scholar 

  • Kuhn, H. (1955). The Hungarian method for solving the assignment problem. Naval Research Logistics Quarterly, 2, 83–97.

    Article  MathSciNet  Google Scholar 

  • Mann, S., & Picard, R. (1997). Video orbits of projective group: An approach to featureless estimation of parameters. IEEE TIP, 6, 1281–1295.

    Google Scholar 

  • Munkres, J. (1957). Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics, 5(1), 32–38.

    Article  MathSciNet  MATH  Google Scholar 

  • Murty, K. (1968). An algorithm for ranking all the assignments in order of increasing cost. Operations Research, 16(3), 682–687.

    Article  MATH  Google Scholar 

  • Nagel, K., & Schreckenberg, M. (1992). A cellular automaton model for freeway traffic. Journal of Physics I: France, 2(12), 2221–2229.

    Article  Google Scholar 

  • Newell, G. (1961). Nonlinear effects in the dynamics of car following. Operations Research, 9(2), 209–229.

    Article  MathSciNet  MATH  Google Scholar 

  • Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62–66.

    Article  MathSciNet  Google Scholar 

  • Perera, A., Srinivas, C., Hoogs, A., Brooksby, G., & Hu, W. (2006). Multi-object tracking through simultaneous long occlusions and split–merge conditions. In CVPR, New York.

  • Pirsiavash, H., Ramanan, D., & Fowlkes, C. (2011). Globally-optimal greedy algorithms for tracking a variable number of objects. In CVPR, Colorado Springs (pp. 1201–1208).

  • Porikli, F., & Pan., P. (2009). Regressed importance sampling on manifolds for efficient object tracking. AVSS, Genoa.

  • Reid, D. (1979). An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control, 24, 843–854.

    Article  Google Scholar 

  • Reilly, V., Idrees, H., & Shah, M. (2010). Detection and tracking of large number of targets in wide area surveillance. In ECCV, Heraklion.

  • Schubert, R., Richter, E., & Wanielik, G. (2008). Comparison and evaluation of advanced motion models for vehicle tracking. In ICIF, Shanghai.

  • Schulz, D., Burgard, W., Fox, D., & Cremers, A. (2001). Tracking multiple moving targets with a mobile robot using particle filters and statistical data association. In ICRA, Seoul.

  • Shafique, K., & Shah, M. (2005). A non-iterative greedy algorithm for multi-frame point correspondence. IEEE TPAMI, 27(1), 51–65.

    Article  Google Scholar 

  • Shalom, Y., & Fortmann, T. (1988). Tracking and data association. London: Academic Press.

    MATH  Google Scholar 

  • Shamos, M., & Hoey, D. (1976). Geometric intersection problems. In SFCS, Houston (pp. 208–215).

  • Song, B., Jeng, T., Staudt, E., & Roy-chowdhury, A. (2010). A stochastic graph evolution framework for robust multi-target tracking. In ECCV, Crete.

  • Stauffer, C., & Grimson, W. (2000). Learning patterns of activity using real-time tracking. IEEE TPAMI, 22(8), 747–757.

    Article  Google Scholar 

  • USAF. (2006). Columbus large image Format dataset. https://www.sdms.afrl.af.mil/datasets/clif2006/.

  • Vezzani, R., Baltieri, D., & Cucchiara, R. (2009). Pathnodes integration of standalone particle filters for people tracking on distributed surveillance systems. In ICIAP, Vietri sul Mare.

  • Wang, G., Xiao, D., & Gu, J. (2008). Review on vehicle detection based on video for traffic surveillance. In ICAL, Qindao (pp. 2961–2966).

  • Xiao, J., Cheng, H., Han, F., & Sawhney, H. (2008). Geo-spatial aerial video processing for scene understanding and object tracking. In CVPR, Anchorage.

  • Xiao, J., Cheng, H., Sawhney, H., & Han, F. (2010). Vehicle detection and tracking in wide field-of-view aerial video. In CVPR, San Francisco.

  • Xing, J., Ai, H., & Lao, S. (2009). Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. In CVPR, Miami.

  • Yang, M., Lv, F., Xu, W., & Gong, Y. (2009). Detection driven adaptive multi-cue integration for multiple human tracking. In ICCV, Kyoto.

  • Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: A survey. ACM Computing Surveys, 38, 1–45.

    Article  Google Scholar 

  • Yin, Z., & Collins, R. (2006). Moving object localization in thermal imagery by forward–backward MHI. In OTCBVS, Kokomo.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering & Computer Science, University of Central Florida, 4000 Central Florida Blvd., Orlando, FL, 32816, USA

    Imran Saleemi & Mubarak Shah

Authors
  1. Imran Saleemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mubarak Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Imran Saleemi.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Cite this article

Saleemi, I., Shah, M. Multiframe Many–Many Point Correspondence for Vehicle Tracking in High Density Wide Area Aerial Videos. Int J Comput Vis 104, 198–219 (2013). https://doi.org/10.1007/s11263-013-0624-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0624-1

Keywords

Search

Navigation