Abstract
Recently, CLEAR and trajectory-based evaluation protocols which generate particular scores such as MOTA and MOTP, etc., are often used in evaluating multiple object tracking (MOT) methods. These scores, indicating how good of tracking methods, seem to be good enough to compare their performances. However, we argue that it is insufficient since failure causes of tracking methods are not discovered. Understanding failure causes will definitely not only help improve their algorithms but also assess merits and demerits of algorithms explicitly. Thus this paper presents Tracking Evaluation Analysis (TEA) by answering the question: “why do tracking failures happen?” TEA comes out as an automatic solution, rather than a conventional way of manually analyzing tracking results, which are notorious for being time-consuming and tedious. In this preliminary version, we demonstrate the validity of TEA by comparing the performances of MOT methods, submitted to MOT 2015 Challenge, tested on TownCentre dataset.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We do not consider all methods in MOT 2015 challenge and only test with TownCentre dataset, because of space limitation.
- 2.
It is trivial to consider the person giving id to the noise.
- 3.
It is also trivial to consider the unmatched track receiving id from a nearby person.
References
Zamir, A.R., Dehghan, A., Shah, M.: GMCP-tracker: global multi-object tracking using generalized minimum clique graphs. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, pp. 343–356. Springer, Heidelberg (2012)
Solera, F., Calderara, S., Cucchiara, R.: Learning to divide and conquer for online multi-target object tracking. In: ICCV, pp. 1–12 (2015)
Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: CVPR, pp. 3457–3464 (2011)
Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. PAMI 36, 58–72 (2014)
Dicle, C., Camps, O., Sznaier, M.: The way they move: tracking multiple targets with similar appearance. In: ICCV, pp. 2304–2311 (2013)
Hamid Rezatofighi, S., Milan, A., Zhang, Z., Shi, Q., Dick, A., Reid, I.: Joint probabilistic data association revisited. In: ICCV, pp. 3047–3055 (2015)
Solera, F., Calderara, S., Cucchiara, R.: Learning to divide and conquer for online multi-target tracking. In: ICCV, pp. 4373–4381 (2015)
Kim, C., Li, F., Ciptadi, A., Rehg, J.M.: Multiple hypothesis tracking revisited. In: ICCV, pp. 4696–4704 (2015)
Yoon, J.H., Yang, M.H., Lim, J., Yoon, K.J.: Bayesian multi-object tracking using motion context from multiple objects. In: WACV, pp. 33–40 (2015)
Geiger, A., Lauer, M., Wojek, C., Stiller, C., Urtasun, R.: 3d traffic scene understanding from movable platforms. PAMI 36, 1012–1025 (2014)
Yang, M., Jia, Y.: Temporal dynamic appearance modeling for online multi-person tracking. arXiv preprint arxiv:1510.02906 (2015)
Anonymous: CFMCMC (2015)
Leal-Taixé, L.: Linear programming on 2D image coordinates (LP2D) (2015)
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. J. Image Video Process. 1, 1–10 (2008)
Li, Y., Huang, C., Nevatia, R.: Learning to associate: hybrid boosted multi-target tracker for crowded scene. In: CVPR, pp. 2953–2960 (2009)
Milan, A., Schindler, K., Roth, S.: Challenges of ground truth evaluation of multi-target tracking. In: CVPR, pp. 735–742 (2013)
Ferryman, J., Ellis, A.: Pets 2010: dataset and challenge. In: AVSS, pp. 143–150 (2010)
Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: towards a benchmark for multi-target tracking. arxiv:1504.01942 [cs] (2015)
Shu, G., Dehghan, A., Oreifej, O., Hand, E., Shah, M.: Part-based multiple-person tracking with partial occlusion handling. In: CVPR, pp. 1815–1821 (2012)
Andriluka, M., Roth, S., Schiele, B.: Monocular 3d pose estimation and tracking by detection. In: CVPR, pp. 623–630 (2010)
Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection-by-tracking. In: CVPR, pp. 1–8 (2008)
Ess, A., Leibe, B., Schindler, K., van Gool, L.: A mobile vision system for robust multi-person tracking. In: CVPR (2008)
Milan, A., Gade, R., Dick, A., Moeslund, T.B., Reid, I.: Improving global multi-target tracking with local updates. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8927, pp. 174–190. Springer, Heidelberg (2015). doi:10.1007/978-3-319-16199-0_13
Zhang, J., Presti, L.L., Sclaroff, S.: Online multi-person tracking by tracker hierarchy. In: AVSS, pp. 379–385 (2012)
Benfold, B., Reid, I.: Guiding visual surveillance by tracking human attention. In: BMVC, pp. 1–11 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Hung, D.H., Tuan, D.A., Khanh, N.N., Hien, T.D., Duong, N.H. (2016). A Multiple Object Tracking Evaluation Analysis Framework. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2016. Lecture Notes in Computer Science(), vol 10072. Springer, Cham. https://doi.org/10.1007/978-3-319-50835-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-50835-1_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50834-4
Online ISBN: 978-3-319-50835-1
eBook Packages: Computer ScienceComputer Science (R0)