Skip to main content
SpringerLink
Log in

Graph-Based Data Association in Multiple Object Tracking: A Survey

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13834))

Included in the following conference series:

Abstract

In Multiple Object Tracking (MOT), data association is a key component of the tracking-by-detection paradigm and endeavors to link a set of discrete object observations across a video sequence, yielding possible trajectories. Our intention is to provide a classification of numerous graph-based works according to the way they measure object dependencies and their footprint on the graph structure they construct. In particular, methods are organized into Measurement-to-Measurement (MtM), Measurement-to-Track (MtT), and Track-to-Track (TtT). At the same time, we include recent Deep Learning (DL) implementations among traditional approaches to present the latest trends and developments in the field and offer a performance comparison. In doing so, this work serves as a foundation for future research by providing newcomers with information about the graph-based bibliography of MOT.

This work was partially supported by the projects NESTOR (H2020-101021851), INFINITY (H2020-883293), and ODYSSEUS (H2020-101021857), funded by the European Commission.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://motchallenge.net.

References

  1. Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection-by-tracking. In: CVPR, pp. 1–8. IEEE (2008)

    Google Scholar 

  2. Berclaz, J., Fleuret, F., Turetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. PAMI 33(9), 1806–1819 (2011)

    Article  Google Scholar 

  3. Brasó, G., Leal-Taixé, L.: Learning a neural solver for multiple object tracking. In: CVPR, pp. 6247–6257. IEEE (2020)

    Google Scholar 

  4. Chong, C.Y.: Graph approaches for data association. In: FUSION, pp. 1578–1585. IEEE (2012)

    Google Scholar 

  5. Chong, C.Y.: An overview of machine learning methods for multiple target tracking. In: FUSION, pp. 1–9. IEEE (2021)

    Google Scholar 

  6. Colson, B., Marcotte, P., Savard, G.: An overview of bilevel optimization. Ann. Oper. Res. 153(1), 235–256 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  7. Dehghan, A., Modiri Assari, S., Shah, M.: GMMCP tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In: CVPR, pp. 4091–4099. IEEE (2015)

    Google Scholar 

  8. Dendorfer, P., et al.: Motchallenge: a benchmark for single-camera multiple target tracking. IJCV 129(4), 845–881 (2021)

    Article  Google Scholar 

  9. Emami, P., Pardalos, P.M., Elefteriadou, L., Ranka, S.: Machine learning methods for data association in multi-object tracking. CSUR 53(4), 1–34 (2020)

    Article  Google Scholar 

  10. Fan, L., et al.: A survey on multiple object tracking algorithm. In: ICIA, pp. 1855–1862. IEEE (2016)

    Google Scholar 

  11. Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: ICML, pp. 1263–1272. PMLR (2017)

    Google Scholar 

  12. Gold, S., Rangarajan, A., et al.: Softmax to softassign: neural network algorithms for combinatorial optimization. Artif. Neural Netw. 2, 381–399 (1996)

    Google Scholar 

  13. He, J., Huang, Z., Wang, N., Zhang, Z.: Learnable graph matching: incorporating graph partitioning with deep feature learning for multiple object tracking. In: CVPR, pp. 5299–5309. IEEE (2021)

    Google Scholar 

  14. Jiang, X., Li, P., Li, Y., Zhen, X.: Graph neural based end-to-end data association framework for online multiple-object tracking. arXiv:1907.05315 (2019)

  15. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv:1609.02907 (2016)

  16. Leal-Taixé, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: siamese CNN for robust target association. In: CVPR Workshops, pp. 33–40. IEEE (2016)

    Google Scholar 

  17. Leal-Taixé, L., Pons-Moll, G., Rosenhahn, B.: Everybody needs somebody: modeling social and grouping behavior on a linear programming multiple people tracker. In: ICCV Workshops, pp. 120–127. IEEE (2011)

    Google Scholar 

  18. Li, J., Gao, X., Jiang, T.: Graph networks for multiple object tracking. In: WACV, pp. 719–728. IEEE (2020)

    Google Scholar 

  19. Li, S., Kong, Y., Rezatofighi, H.: Learning of global objective for network flow in multi-object tracking. In: CVPR, pp. 8855–8865 (2022)

    Google Scholar 

  20. Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Kim, T.K.: Multiple object tracking: a literature review. Artif. Intell. 293, 103448 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  21. Ma, C., et al.: Deep association: end-to-end graph-based learning for multiple object tracking with conv-graph neural network. In: ICMR, pp. 253–261. ACM (2019)

    Google Scholar 

  22. Papakis, I., Sarkar, A., Karpatne, A.: GCNNMatch: graph convolutional neural networks for multi-object tracking via sinkhorn normalization. arXiv:2010.00067 (2020)

  23. Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR, pp. 1201–1208. IEEE (2011)

    Google Scholar 

  24. Rangesh, A., Maheshwari, P., Gebre, M., Mhatre, S., Ramezani, V., Trivedi, M.M.: TrackMPNN: a message passing graph neural architecture for multi-object tracking. arXiv:2101.04206 (2021)

  25. Roshan Zamir, A., Dehghan, A., Shah, M.: GMCP-tracker: global multi-object tracking using generalized minimum clique graphs. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7573, pp. 343–356. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33709-3_25

    Chapter  Google Scholar 

  26. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. Neural Netw. 20(1), 61–80 (2008)

    Article  Google Scholar 

  27. Schulter, S., Vernaza, P., Choi, W., Chandraker, M.: Deep network flow for multi-object tracking. In: CVPR, pp. 6951–6960. IEEE (2017)

    Google Scholar 

  28. Shen, H., Huang, L., Huang, C., Xu, W.: Tracklet association tracker: an end-to-end learning-based association approach for multi-object tracking. arXiv:1808.01562 (2018)

  29. Singh, T., Vishwakarma, D.K.: Human activity recognition in video benchmarks: a survey. In: Advances in Signal Processing and Communication, pp. 247–259 (2019)

    Google Scholar 

  30. Sun, Z., Chen, J., Chao, L., Ruan, W., Mukherjee, M.: A survey of multiple pedestrian tracking based on tracking-by-detection framework. CSVT 31(5), 1819–1833 (2020)

    Google Scholar 

  31. Tang, S., Andres, B., Andriluka, M., Schiele, B.: Subgraph decomposition for multi-target tracking. In: CVPR, pp. 5033–5041. IEEE (2015)

    Google Scholar 

  32. Tang, S., Andres, B., Andriluka, M., Schiele, B.: Multi-person tracking by multicut and deep matching. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 100–111. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_8

    Chapter  Google Scholar 

  33. Tang, S., Andriluka, M., Andres, B., Schiele, B.: Multiple people tracking by lifted multicut and person re-identification. In: CVPR, pp. 3539–3548. IEEE (2017)

    Google Scholar 

  34. Wang, B., Wang, G., Chan, K.L., Wang, L.: Tracklet association by online target-specific metric learning and coherent dynamics estimation. PAMI 39(3), 589–602 (2016)

    Article  Google Scholar 

  35. Wang, B., et al.: Joint learning of siamese CNNs and temporally constrained metrics for tracklet association. arXiv:1605.04502 (2016)

  36. Wang, Y., Kitani, K., Weng, X.: Joint object detection and multi-object tracking with graph neural networks. In: ICRA, pp. 13708–13715. IEEE (2021)

    Google Scholar 

  37. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. Neural Netw. Learn. Syst. 32(1), 4–24 (2020)

    Article  MathSciNet  Google Scholar 

  38. Xu, Y., Zhou, X., Chen, S., Li, F.: Deep learning for multiple object tracking: a survey. IET Comput. Vision 13(4), 355–368 (2019)

    Article  Google Scholar 

  39. Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: CVPR, pp. 1–8. IEEE (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information Technologies Institute, Centre for Research and Technology Hellas, Thermi-Thessaloniki, Greece

    Despoina Touska, Konstantinos Gkountakos, Theodora Tsikrika, Konstantinos Ioannidis, Stefanos Vrochidis & Ioannis Kompatsiaris

Authors
  1. Despoina Touska
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantinos Gkountakos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Theodora Tsikrika
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Konstantinos Ioannidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stefanos Vrochidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ioannis Kompatsiaris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Despoina Touska .

Editor information

Editors and Affiliations

  1. University of Bergen, Bergen, Norway

    Duc-Tien Dang-Nguyen

  2. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  3. Radboud University Nijmegen, Nijmegen, The Netherlands

    Martha Larson

  4. Dublin City University, Dublin, Ireland

    Alan F. Smeaton

  5. University of Amsterdam, Amsterdam, The Netherlands

    Stevan Rudinac

  6. National Institute of Information and Communications Technology, Tokyo, Japan

    Minh-Son Dao

  7. Department of Information Science and Media Studies, University of Bergen, Bergen, Norway

    Christoph Trattner

  8. La Trobe University, Melbourne, VIC, Australia

    Phoebe Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Touska, D., Gkountakos, K., Tsikrika, T., Ioannidis, K., Vrochidis, S., Kompatsiaris, I. (2023). Graph-Based Data Association in Multiple Object Tracking: A Survey. In: Dang-Nguyen, DT., et al. MultiMedia Modeling. MMM 2023. Lecture Notes in Computer Science, vol 13834. Springer, Cham. https://doi.org/10.1007/978-3-031-27818-1_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-27818-1_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-27817-4

  • Online ISBN: 978-3-031-27818-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation