Skip to main content
SpringerLink
Log in

Fast online multi-target multi-camera tracking for vehicles

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-target multi-camera tracking (MTMCT) for vehicles, which aims to track multiple vehicles across multi-camera environments, is crucial in surveillance or intelligent transportation systems due to its broad applicability in real situations. However, the high inter-class similarity of vehicles and also their high intra-class variability due to the varying perspective, lighting, and video quality of each camera make it significantly challenging. Various offline approaches have been proposed and dominated the field with further advantages over the online strategy, but they are hardly adopted in real-world applications that usually require an online operation. In this paper, we propose a novel fast online MTMCT algorithm for vehicles considering better applicability in real applications. During the MTMCT, we actively reflect online MTSCT results, which is more reliable than clustering results in the multi-camera domain, on top of the object detection and feature extraction. To do so, we can effectively reduce the ID switches of the tracks and computational costs by decreasing the number of feature comparisons. As a result, we achieve 77.3 IDF1 on the S02 scenario of the CityFlow dataset with 0.012 seconds of tracking speed with four camera inputs. The source code is released at https://github.com/kamkyu94/Fast_Online_MTMCT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1

Similar content being viewed by others

Data Availability

The dataset used during this study is available from the first author upon reasonable request.

References

  1. Aharon N, Orfaig R, Bobrovsky BZ (2022) Bot-sort: Robust associations multi-pedestrian tracking. arXiv:2206.14651. https://doi.org/10.48550/arXiv.2206.14651

  2. Chang MC, Wei J, Zhu ZA et al (2019) Ai city challenge 2019 - city-scale video analytics for smart transportation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 99–108

  3. Deng J, Dong W, Socher R et al (2009) Imagenet: A large-scale hierarchical image database. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848

  4. Du Y, Wan J, Zhao Y et al (2021) Giaotracker: A comprehensive framework for mcmot with global information and optimizing strategies in visdrone 2021. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp 2809–2819. https://doi.org/10.1109/ICCVW54120.2021.00315

  5. Du Y, Zhao Z, Song Y et al (2023) Strongsort: Make deepsort great again. IEEE Trans Multimed. https://doi.org/10.1109/TMM.2023.3240881

    Article  Google Scholar 

  6. Dunn JC (1974) Well-separated clusters and optimal fuzzy partitions. J Cybern 4(1):95–104. https://doi.org/10.1080/01969727408546059

    Article  MathSciNet  MATH  Google Scholar 

  7. Guo Y, Liu Z, Luo H et al (2022) Multi-person multi-camera tracking for live stream videos based on improved motion model and matching cascade. Neurocomput 492:561–571. https://doi.org/10.1016/j.neucom.2021.12.047

    Article  Google Scholar 

  8. He K, Gkioxari G, Dollár P et al (2017) Mask r-cnn. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 2980–2988. https://doi.org/10.1109/ICCV.2017.322

  9. He Y, Wei X, Hong X et al (2020) Multi-target multi-camera tracking by tracklet-to-target assignment. IEEE Trans Image Process 29:5191–5205. https://doi.org/10.1109/TIP.2020.2980070

    Article  MathSciNet  MATH  Google Scholar 

  10. He Z, Lei Y, Bai S et al (2019) Multi-camera vehicle tracking with powerful visual features and spatial-temporal cue. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 203–212

  11. Hou Y, Du H, Zheng L (2019) A locality aware city-scale multi-camera vehicle tracking system. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 167–174

  12. Hou Y, Wang Z, Wang S et al (2022) Adaptive affinity for associations in multi-target multi-camera tracking. IEEE Trans Image Process 31:612–622. https://doi.org/10.1109/TIP.2021.3131936

    Article  Google Scholar 

  13. Hsu HM, Huang TW, Wang G et al (2019) Multi-camera tracking of vehicles based on deep features re-id and trajectory-based camera link models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 416–424

  14. Hsu HM, Cai J, Wang Y et al (2021) Multi-target multi-camera tracking of vehicles using metadata-aided re-id and trajectory-based camera link model. IEEE Trans Image Process 30:5198–5210. https://doi.org/10.1109/TIP.2021.3078124

    Article  Google Scholar 

  15. Huynh SV, Nguyen NH, Nguyen NT et al (2021) A strong baseline for vehicle re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 4147–4154

  16. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Proceedings of the International Conference for Learning Representations

  17. Lin TY, Maire M, Belongie S et al (2014) Microsoft coco: Common objects in context. In: Proceedings of the European Conference on Computer Vision, pp 740–755. https://doi.org/10.1007/978-3-319-10602-1_48

  18. Liu X, Liu W, Mei T et al (2018) Provid: Progressive and multimodal vehicle reidentification for large-scale urban surveillance. IEEE Trans Multimed 20(3):645–658. https://doi.org/10.1109/TMM.2017.2751966

    Article  Google Scholar 

  19. Liu Z, Shang Y, Li T et al (2023) Robust multi-drone multi-target tracking to resolve target occlusion: A benchmark. IEEE Trans Multimed 25:1462–1476. https://doi.org/10.1109/TMM.2023.3234822

    Article  Google Scholar 

  20. Luna E, Miguel JCS, Martínez JM et al (2022a) Graph convolutional network for multi-target multi-camera vehicle tracking. arXiv:2211.15538. https://doi.org/10.48550/ARXIV.2211.15538

  21. Luna E, SanMiguel JC, Martínez JM et al (2022) Online clustering-based multi-camera vehicle tracking in scenarios with overlapping fovs. Multimed Tools Appl 81(5):7063–7083. https://doi.org/10.1007/s11042-022-11923-2

    Article  Google Scholar 

  22. Luo H, Jiang W, Gu Y et al (2020) A strong baseline and batch normalization neck for deep person re-identification. IEEE Trans Multimed 22(10):2597–2609. https://doi.org/10.1109/TMM.2019.2958756

    Article  Google Scholar 

  23. Munkres J (1957) Algorithms for the assignment and transportation problems. J Soc Ind Appl Math 5:32–38. https://doi.org/10.1137/0105003

  24. Pan X, Luo P, Shi J et al (2018) Two at once: Enhancing learning and generalization capacities via ibn-net. In: Proceedings of the European Conference on Computer Vision, pp 484–500. https://doi.org/10.1007/978-3-030-01225-0_29

  25. Quach KG, Nguyen P, Le H et al (2021) Dyglip: A dynamic graph model with link prediction for accurate multi-camera multiple object tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 13,779–13,788. https://doi.org/10.1109/CVPR46437.2021.01357

  26. Radenović F, Tolias G, Chum O (2019) Fine-tuning cnn image retrieval with no human annotation. IEEE Trans Pattern Anal Mach Intell 41(7):1655–1668. https://doi.org/10.1109/TPAMI.2018.2846566

    Article  Google Scholar 

  27. Ristani E, Tomasi C (2018) Features for multi-target multi-camera tracking and re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6036–6046. https://doi.org/10.1109/CVPR.2018.00632

  28. Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 15(56):1929–1958

    MathSciNet  MATH  Google Scholar 

  29. Tang Z, Hwang JN (2019) Moana: An online learned adaptive appearance model for robust multiple object tracking in 3d. IEEE Access 7:31,934–31,945. https://doi.org/10.1109/ACCESS.2019.2903121

  30. Tang Z, Naphade M, Liu MY et al (2019) Cityflow: A city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8797–8806. https://doi.org/10.1109/CVPR.2019.00900

  31. Wang CY, Bochkovskiy A, Liao HYM (2023) Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7464–7475

  32. Wojke N, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. In: IEEE International Conference on Image Processing, pp 3645–3649. https://doi.org/10.1109/ICIP.2017.8296962

  33. Xu J, Bo C, Wang D (2021) A novel multi-target multi-camera tracking approach based on feature grouping. Comput Electr Eng 92(107):153. https://doi.org/10.1016/j.compeleceng.2021.107153

    Article  Google Scholar 

  34. You S, Yao H, Xu C (2021) Multi-target multi-camera tracking with optical-based pose association. IEEE Trans Circ Syst Video Technol 31(8):3105–3117. https://doi.org/10.1109/TCSVT.2020.3036467

    Article  Google Scholar 

  35. Zhang Y, Wang C, Wang X et al (2021) Fairmot: On the fairness of detection and re-identification in multiple object tracking. Int J Comput Vision 129(11):3069–3087. https://doi.org/10.1007/s11263-021-01513-4

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by LIG Nex1.Co., Ltd, originally funded by DAPA and ADD (UC190031FD).

Author information

Authors and Affiliations

  1. The School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea

    Kyujin Shim, Kangwook Ko, Jubi Hwang & Changick Kim

  2. EO/IR Systems R &D Lab, LIG Nex1 Co. Ltd, Yongin, 16911, Republic of Korea

    Hyunsung Jang

Authors
  1. Kyujin Shim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kangwook Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jubi Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hyunsung Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Changick Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changick Kim.

Ethics declarations

Conflict of interest

The authors declare that there are no competing interests related to this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shim, K., Ko, K., Hwang, J. et al. Fast online multi-target multi-camera tracking for vehicles. Appl Intell 53, 28994–29004 (2023). https://doi.org/10.1007/s10489-023-05081-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05081-7

Keywords

Search

Navigation