Skip to main content
SpringerLink
Log in

Unsupervised classification of erroneous video object trajectories

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The paper proposes a method to detect failures in object tracking. Detection is done with the help of two types of errors, namely jump and stop errors. Jump errors occur when an abrupt change in object’s motion is observed, whereas stop errors occur when a moving object remains stationary for longer duration at any point. In our framework, moving objects are first tracked using well-known trackers and their trajectories are obtained. Discrepancies between trajectories are measured. We have shown that the proposed method can be reliable for detection of tracking failures. This can help to find error-free trajectories that are essential in various computer vision tasks. We have also shown that the tracking performance can be further improved while processing the output trajectories without much knowledge about the underlying tracking algorithms. The effect of tracking failure is investigated to identify erroneous trajectories. It has been observed that when a tracker fails, velocity profile of the moving object usually changes significantly. Based on this hypothesis, erroneous trajectories are detected and a set of error-free trajectories are marked and grouped. Two recently proposed tracking algorithms, namely real-time compressive tracker (CT) and real-time L1-tracker (L1APG), have been used to track the objects. We have tested our framework on five publicly available datasets containing more than 300 trajectories. Our experiments reveal that average classification rate of erroneous trajectories can be as high as 80.4% when objects are tracked using L1APG tracker. Accuracy can be as high as 81.2% when applied on trajectories obtained using CT tracker. Average accuracy of tracking increases significantly (19.2% with respect to L1APG tracker and 24.8% with respect to CT tracker) when the decision is taken using a fused framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  • Adam A, Rivlin E, Shimshoni I (2006) Robust fragments-based tracking using the integral histogram. In: Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06). IEEE, vol 1, pp 798–805

  • Ahmed A, Dogra D, Kar S, Kim B, Hill P, Bhaskar H (2016) Localization of region of interest in surveillance scene. Multimed Tools Appl. doi:10.1007/s11042-016-3762-y

    Google Scholar 

  • Ali S, Shah M (2007) A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–6

  • Andriyenko A, Schindler K (2011) Multi-target tracking by continuous energy minimization. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1265–1272

  • Babenko YMH B, Belongie S (2011) Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Intell 33(8):1619–1632

    Article  Google Scholar 

  • Bae SH, Yoon KJ (2014) Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1218–1225

  • Bailer C, Pagani A, Stricker D (2014) A superior tracking approach: Building a strong tracker through fusion. In: Computer vision–ECCV 2014. Springer, pp 170–185

  • Bao C, Wu Y, Ling H, Ji H (2012) Real time robust l1 tracker using accelerated proximal gradient approach. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1830–1837

  • Benfold B, Reid I (2011) Stable multi-target tracking in real-time surveillance video. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 3457–3464

  • Bernardin K, Stiefelhagen R (2008) Evaluating multiple object tracking performance: the clear mot metrics. J Image Video Process 2008:1

    Article  Google Scholar 

  • Bibby C, Reid I (2008) Robust real-time visual tracking using pixel-wise posteriors. In: Computer vision–ECCV 2008. Springer, pp 831–844

  • Birchfield S (2007) Klt: An implementation of the Kanade–Lucas–Tomasi feature tracker

  • Comaniciu D, Ramesh V, Meer P (2000) Real-time tracking of non-rigid objects using mean shift. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 2:142–149

    Google Scholar 

  • Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577

    Article  Google Scholar 

  • Dehghan A, Idrees H, Zamir A, Shah M (2014) Automatic detection and tracking of pedestrians in videos with various crowd densities. In: Proceedings of the pedestrian and evacuation dynamics, pp 3–19

  • Doğan S, Temiz M, Külür S (2010) Real time speed estimation of moving vehicles from side view images from an uncalibrated video camera. Sensors 10(5):4805–4824

    Article  Google Scholar 

  • Dogra D, Ahmed A, Bhaskar H (2015a) Interest area localization using trajectory analysis in surveillance scenes. In: Proceedings of the 10th international conference on computer vision theory and applications, pp 478–485

  • Dogra D, Reddy R, Subramanyam K, Ahmed A, Bhaskar H (2015b) Scene representation and anomalous activity detection using weighted region association graph. In: Proceedings of the 10th international conference on computer vision theory and applications, pp 104–112

  • Dogra D, Ahmed A, Bhaskar H (2016) Smart video summarization using mealy machine-based trajectory modelling for surveillance applications. Multimed Tools Appl 75:6373–6401

    Article  Google Scholar 

  • Doyle D, Jennings A, Black J (2014) Optical flow background estimation for real-time pan/tilt camera object tracking. Measurement 48:195–207

    Article  Google Scholar 

  • Ekin A, Tekalp A, Mehrotra R (2003) Automatic soccer video analysis and summarization. IEEE Trans Image Process 12(7):796–807

    Article  Google Scholar 

  • Gall J, Yao A, Razavi N, Gool LV, Lempitsky V (2011) Hough forests for object detection, tracking, and action recognition. IEEE Trans Pattern Anal Mach Intell 33(11):2188–2202

    Article  Google Scholar 

  • Hua Y, Alahari K, Schmid C (2015) Online object tracking with proposal selection. In: Proceedings of the IEEE international conference on computer vision, pp 3092–3100

  • Jurie F, Dhome M (2001) Real time tracking of 3d objects with occultations. In: Proceedings of international conference on image processing. IEEE, vol 1, pp 413–416

  • Kaempchen N, Buehler M, Dietmayer K (2005) Feature-level fusion for free-form object tracking using laser scanner and video. In: Intelligent vehicles symposium, 2005. Proceedings. IEEE, pp 453–458

  • Khan M, Valstar M, Pridmore T (2013) A multiple motion model tracker handling occlusion and rapid motion variation. In: British machine vision workshop

  • Kim S, Kwak S, Feyereisl J, Han B (2012) Online multi-target tracking by large margin structured learning. In: Asian conference on computer vision. Springer, pp 98–111

  • Lalonde M, Beaulieu M, Gagnon L (2001) Fast and robust optic disc detection using pyramidal decomposition and hausdorff-based template matching. IEEE Trans Med Imaging 20(11):1193–1200

    Article  Google Scholar 

  • Lan X, Ma A, Yuen P (2014) Multi-cue visual tracking using robust feature-level fusion based on joint sparse representation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1194–1201

  • Leibe B, Schindler K, Gool L (2007) Coupled detection and trajectory estimation for multi-object tracking. In: IEEE 11th international conference on computer vision (ICCV), IEEE, pp 1–8

  • Li F, Cao T, Song Z, Wang W (2015) Object tracking based on locality-constrained linear coding joint sparse representation appearance model. In: IEEE 16th international conference on communication technology (ICCT). IEEE, pp 892–896

  • Li X, Hu W, Shen C, Zhang Z, Dick A, Hengel A (2013) A survey of appearance models in visual object tracking. ACM Trans Intell Syst Technol (TIST) 4(4):58

    Google Scholar 

  • Li Y, Huang C, Nevatia R (2009) Learning to associate: hybrid boosted multi-target tracker for crowded scene. In: IEEE conference on computer vision and pattern recognition, 2009 (CVPR, 2009). IEEE, pp 2953–2960

  • Long C, Hua G, Kapoor A (2015) A joint gaussian process model for active visual recognition with expertise estimation in crowdsourcing. Int J Comput Vis. doi:10.1007/s11263-015-0834-9

    Google Scholar 

  • Ma L, Lu J, Feng J, Zhou J (2015) Multiple feature fusion via weighted entropy for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 3128–3136

  • Mariano V, Min J, Park J, Kasturi R, Mihalcik D, Li H, Doermann D, Drayer T (2002) Performance evaluation of object detection algorithms. In: Proceedings of 16th international conference on pattern recognition, 2002. IEEE, vol 3, pp 965–969

  • Martín R, Martínez J (2014) Evaluation of bounding box level fusion of single target video object trackers. In: Hybrid artificial intelligence systems. Springer, pp 200–210

  • MS Temiz SK, Dogan S (2012) Real time speed estimation from monocular video. In: International archives of the photogrammetry, remote sensing and spatial information sciences, pp 427–432

  • Nummiaro K, Koller-Meier E, Gool LV (2002) Object tracking with an adaptive color-based particle filter. In: Pattern recognition. Springer, pp 353–360

  • Ogiela L, Ogiela MR (2015) Comparison of cognitive information systems supporting management tasks. In: 2015 international conference on intelligent networking and collaborative systems (INCOS). IEEE, pp 49–56

  • Papadourakis V, Argyros A (2010) Multiple objects tracking in the presence of long-term occlusions. Comput Vis Image Underst 114(7):835–846

    Article  Google Scholar 

  • Pettersen S, Johansen D, Oth (2014) Soccer video and player position dataset. In: Proceedings of the 5th ACM multimedia systems conference, pp 18–23

  • Power D, Sharda R, Burstein F (2015) Decision support systems. Wiley Online Library, London

    Book  Google Scholar 

  • Ristic B, Arulampalam S, Gordon N (2004) Beyond the kalman filter. IEEE Aerosp Electron Syst Mag 19(7):37–38

    Article  MATH  Google Scholar 

  • Rousseeuw P (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65

    Article  MATH  Google Scholar 

  • Saini R, Ahmed A, Dogra DP, Roy PP (2017a) Classification of object trajectories represented by high-level features using unsupervised learning. In: Proceedings of international conference on computer vision and image processing. Springer, pp 273–284

  • Saini R, Ahmed A, Dogra DP, Roy PP (2017b) Surveillance scene segmentation based on trajectory classification using supervised learning. In: Proceedings of international conference on computer vision and image processing. Springer, pp 261–271

  • Schoepflin T, Dailey D (2003) Dynamic camera calibration of roadside traffic management cameras for vehicle speed estimation. IEEE Trans Intell Transp Syst 4(2):90–98

    Article  Google Scholar 

  • Shah M, Javed O, Shafique K (2007) Automated visual surveillance in realistic scenarios. IEEE Multimed 14(1):30–39

    Article  Google Scholar 

  • Shao J, Loy C, Wang X (2014) Scene-independent group profiling in crowd. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2227–2234

  • Shin J, Kim S, Kang S, Lee S, Paik J, Abidi B, Abidi M (2005) Optical flow-based real-time object tracking using non-prior training active feature model. Real-Time Imaging 11(3):204–218

    Article  Google Scholar 

  • Smeulders A, Chu D, Cucchiara R, Calderara S, Dehghan A, Shah M (2014) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468

    Article  Google Scholar 

  • Stauffer C, Grimson W (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757

    Article  Google Scholar 

  • Takala V, Pietikainen M (2007) Multi-object tracking using color, texture and motion. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1–7

  • Tang F, Brennan S, Zhao Q, Tao H (2007) Co-tracking using semi-supervised support vector machines. In: IEEE 11th international conference on computer vision (ICCV). IEEE, pp 1–8

  • Wang X, Ma K, Ng G, Grimson W (2011) Trajectory analysis and semantic region modeling using nonparametric hierarchical bayesian models. Int J Comput Vis 95(3):287–312

    Article  Google Scholar 

  • Wu H, Sankaranarayanan A, Chellappa R (2010) Online empirical evaluation of tracking algorithms. IEEE Trans Pattern Anal Mach Intell 32(8):1443–1458

    Article  Google Scholar 

  • Wu Y, Lim J, Yang M (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848

    Article  Google Scholar 

  • Xiang Y, Alahi A, Savarese S (2015) Learning to track: online multi-object tracking by decision making. In: Proc IEEE Int Conf Comput Vis, pp 4705–4713

  • Yilmaz A (2007) Object tracking by asymmetric kernel mean shift with automatic scale and orientation selection. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–6

  • Yilmaz A, Li X, Shah M (2004) Contour-based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans Pattern Anal Mach Intell 26(11):1531–1536

    Article  Google Scholar 

  • Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):13

    Article  Google Scholar 

  • Zhang K, Zhang L, Yang M (2012) Real-time compressive tracking. In: Proceedings of the European conference on computer vision, pp 864–877

  • Zhong Y, Jain A, Dubuisson-Jolly M (2000) Object tracking using deformable templates. IEEE Trans Pattern Anal Mach Intell 22(5):544–549

    Article  Google Scholar 

  • Zhou B, Wang X, Tang X (2012) Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2871–2878

  • Zhou B, Tang X, Zhang H, Wang X (2014a) Measuring crowd collectiveness. IEEE Trans Pattern Anal Mach Intell 36(8):1586–1599

    Article  Google Scholar 

  • Zhou B, Tang X, Wang X (2015) Learning collective crowd behaviors with dynamic pedestrian-agents. Int J Comput Vis 111(1):50–68

    Article  Google Scholar 

  • Zhou H, Yuan Y, Shi C (2009) Object tracking using sift features and mean shift. Comput Vis Image Underst 113(3):345–352

    Article  Google Scholar 

  • Zhou Y, Rao C, Lu Q, Bai X, Liu W (2011) Multiple feature fusion for object tracking. In: Intelligent science and intelligent data engineering. Springer, pp 145–152

  • Zhou Z, Wu D, Peng X, Zhu Z, Luo K (2014b) Object tracking based on camshift with multi-feature fusion. J Softw 9(1):147–153

    Google Scholar 

  • Zivkovic Z, Krose B (2004) An em-like algorithm for color-histogram-based object tracking. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition (CVPR). IEEE, vol 1, pp I–798

Download references

Acknowledgements

This study has not been funded by any agency.

Author information

Authors and Affiliations

  1. Haldia Institute of Technology, Haldia, India

    Sk. Arif Ahmed

  2. Indian Institute of Technology, Bhubaneswar, India

    Debi Prosad Dogra

  3. National Institute of Technology, Durgapur, India

    Samarjit Kar

  4. Indian Institute of Technology, Roorkee, India

    Partha Pratim Roy

Authors
  1. Sk. Arif Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debi Prosad Dogra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Samarjit Kar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Partha Pratim Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sk. Arif Ahmed.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest regarding the publication of this paper.

Ethical approval

This article does not contain any studies on human participants or animals.

Informed consent

Consent was obtained from all volunteers/individual participants included in the study.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, S.A., Dogra, D.P., Kar, S. et al. Unsupervised classification of erroneous video object trajectories. Soft Comput 22, 4703–4721 (2018). https://doi.org/10.1007/s00500-017-2656-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-017-2656-x

Keywords

Search

Navigation