Skip to main content
Springer Nature Link
Log in

Multi-object tracking in UAVs with feature fusion distribution and occlusion awareness

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Multi-object tracking (MOT) in unmanned aerial vehicles (UAVs) is a crucial computer vision task with diverse applications in both military and civilian domains. However, the unique characteristics of UAVs, such as motion uncertainty and sudden changes in viewpoints, lead to objects with scale variance, occlusion, dense distribution, and frequent appearance and disappearance in the image, posing significant challenges in MOT in UAVs. In this paper, we address these issues by proposing two novel techniques: Feature Fusion Distribution Network (FFDN) and Occlusion-Aware Prediction and Association (OAPA), which are integrated into a new MOT algorithm named OATrack. The FFDN aims to improve object detection by optimizing the fusion of multi-scale features within the detection network, especially for densely distributed and different sized objects. The OAPA aims to enhance the accuracy and robustness of prediction and association for objects lost due to occlusion, thus addressing the issue of occlusion in UAV scenes. Experiments are conducted on Visdrone2019 and UAVDT, and the results clearly demonstrate the effectiveness and superiority of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

References

  1. Xue, Y., Jin, G., Shen, T., Tan, L., Wang, N., Gao, J., Wang, L.: Smalltrack: wavelet pooling and graph enhanced classification for uav small object tracking. IEEE Trans. Geosci. Remote Sens. (2023)

  2. Xue, Y., Jin, G., Shen, T., Tan, L., Wang, L.: Template-guided frequency attention and adaptive cross-entropy loss for uav visual tracking. Chin. J. Aeronaut. 36(9), 299–312 (2023)

    Article  MATH  Google Scholar 

  3. Xue, Y., Jin, G., Shen, T., Tan, L., Yang, J., Hou, X.: Mobiletrack: Siamese efficient mobile network for high-speed uav tracking. IET Image Proc. 16(12), 3300–3313 (2022)

    Article  Google Scholar 

  4. Wang, J., Meng, C., Deng, C., Wang, Y.: Learning convolutional self-attention module for unmanned aerial vehicle tracking. SIViP 17(5), 2323–2331 (2023)

    Article  MATH  Google Scholar 

  5. Xue, Y., Shen, T., Jin, G., Tan, L., Wang, N., Wang, L., Gao, J.: Handling occlusion in uav visual tracking with query-guided redetection. IEEE Trans. Instrum. Meas. (2024)

  6. Wang, Z., Zheng, L., Liu, Y., Li, Y., Wang, S.: Towards Real-Time Multi-object Tracking, pp. 107–122. Springer (2020)

  7. Zhang, Y., Wang, C., Wang, X., Zeng, W., Liu, W.: Fairmot: on the fairness of detection and re-identification in multiple object tracking. Int. J. Comput. Vis. 129, 3069–3087 (2021)

    Article  MATH  Google Scholar 

  8. Liu, S., Li, X., Lu, H., He, Y.: Multi-object tracking meets moving uav, pp. 8876–8885 (2022)

  9. Liang, C., Zhang, Z., Zhou, X., Li, B., Zhu, S., Hu, W.: Rethinking the competition between detection and reid in multiobject tracking. IEEE Trans. Image Process. 31, 3182–3196 (2022)

    Article  MATH  Google Scholar 

  10. Li, J., Ding, Y., Wei, H.-L., Zhang, Y., Lin, W.: Simpletrack: rethinking and improving the jde approach for multi-object tracking. Sensors 22(15), 5863 (2022)

    Article  MATH  Google Scholar 

  11. Wu, H., Nie, J., He, Z., Zhu, Z., Gao, M.: One-shot multiple object tracking in uav videos using task-specific fine-grained features. Remote Sens. 14(16), 3853 (2022)

    Article  MATH  Google Scholar 

  12. Xiao, C., Cao, Q., Zhong, Y., Lan, L., Zhang, X., Cai, H., Luo, Z.: Enhancing online uav multi-object tracking with temporal context and spatial topological relationships. Drones 7(6), 389 (2023)

    Article  MATH  Google Scholar 

  13. Wu, H., Nie, J., Zhu, Z., He, Z., Gao, M.: Learning task-specific discriminative representations for multiple object tracking. Neural Comput. Appl. 35(10), 7761–7777 (2023)

    Article  MATH  Google Scholar 

  14. Yao, M., Wang, J., Peng, J., Chi, M., Liu, C.: Folt: fast multiple object tracking from uav-captured videos based on optical flow, pp. 3375–3383 (2023)

  15. Stadler, D., Sommer, L.W., Beyerer, J.: Pas tracker: Position-, appearance-and size-aware multi-object tracking in drone videos. In: Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16, pp. 604–620 (2020). Springer

  16. Wang, P., Wang, Y., Li, D.: Dronemot: Drone-based multi-object tracking considering detection difficulties and simultaneous moving of drones and objects. In: 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 7397–7404 (2024). IEEE

  17. Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors, pp. 7464–7475 (2023)

  18. Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J.: Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 (2021)

  19. Zhao, Y., Lv, W., Xu, S., Wei, J., Wang, G., Dang, Q., Liu, Y., Chen, J.: Detrs beat yolos on real-time object detection. arXiv preprint arXiv:2304.08069 (2023)

  20. Wen, L., Zhu, P., Du, D., Bian, X., Ling, H., Hu, Q., Zheng, J., Peng, T., Wang, X., Zhang, Y., et al.: Visdrone-mot2019: The vision meets drone multiple object tracking challenge results (2019)

  21. Du, D., Qi, Y., Yu, H., Yang, Y., Duan, K., Li, G., Zhang, W., Huang, Q., Tian, Q.: The unmanned aerial vehicle benchmark: object detection and tracking, pp. 370–386 (2018)

  22. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection, pp. 2117–2125 (2017)

  23. Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation, pp. 8759–8768 (2018)

  24. Tan, M., Pang, R., Le, Q.V.: Efficientdet: scalable and efficient object detection, pp. 10781–10790 (2020)

  25. Zhao, Q., Sheng, T., Wang, Y., Tang, Z., Chen, Y., Cai, L., Ling, H.: M2det: a single-shot object detector based on multi-level feature pyramid network. Proc. AAAI Conf. Artif. Intell. 33(01), 9259–9266 (2019)

    Google Scholar 

  26. Yang, G., Lei, J., Zhu, Z., Cheng, S., Feng, Z., Liang, R.: Afpn: asymptotic feature pyramid network for object detection, pp. 2184–2189 (2023). IEEE

  27. Quan, Y., Zhang, D., Zhang, L., Tang, J.: Centralized feature pyramid for object detection. IEEE Trans. Image Process. (2023)

  28. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking, pp. 3464–3468 (2016). IEEE

  29. Girshick, R.: Fast r-cnn, pp. 1440–1448 (2015)

  30. Welch, G., Bishop, G., et al.: An introduction to the kalman filter (1995)

  31. Kuhn, H.W.: The hungarian method for the assignment problem. Naval Res. Logist. Q. 2(1–2), 83–97 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  32. Zhang, Y., Sun, P., Jiang, Y., Yu, D., Weng, F., Yuan, Z., Luo, P., Liu, W., Wang, X.: Bytetrack: multi-object tracking by associating every detection box, pp. 1–21 (2022). Springer

  33. Cao, J., Pang, J., Weng, X., Khirodkar, R., Kitani, K.: Observation-centric sort: rethinking sort for robust multi-object tracking, pp. 9686–9696 (2023)

  34. Yang, M., Han, G., Yan, B., Zhang, W., Qi, J., Lu, H., Wang, D.: Hybrid-sort: weak cues matter for online multi-object tracking. Proc. AAAI Conf. Artif. Intell. 38(7), 6504–6512 (2024)

    MATH  Google Scholar 

  35. Su, Y., Sun, R., Shu, X., Zhang, Y., Wu, Q.: Occlusion-aware detection and re-id calibrated network for multi-object tracking. arXiv preprint arXiv:2308.15795 (2023)

  36. Liu, Q., Chen, D., Chu, Q., Yuan, L., Liu, B., Zhang, L., Yu, N.: Online multi-object tracking with unsupervised re-identification learning and occlusion estimation. Neurocomputing 483, 333–347 (2022)

    Article  MATH  Google Scholar 

  37. Wang, Q., Zheng, Y., Pan, P., Xu, Y.: Multiple object tracking with correlation learning, pp. 3876–3886 (2021)

  38. Jiang, M., Zhou, C., Kong, J.: Aoh: online multiple object tracking with adaptive occlusion handling. IEEE Signal Process. Lett. 29, 1644–1648 (2022)

    Article  MATH  Google Scholar 

  39. Aharon, N., Orfaig, R., Bobrovsky, B.-Z.: Bot-sort: Robust associations multi-pedestrian tracking. arXiv preprint arXiv:2206.14651 (2022)

  40. Du, Y., Zhao, Z., Song, Y., Zhao, Y., Su, F., Gong, T., Meng, H.: Strongsort: make deepsort great again. IEEE Trans. Multimedia (2023)

Download references

Author information

Authors and Affiliations

  1. Beihang University, Beijing, 100191, China

    Yuchen Wang & Wei Zhao

  2. Beijing Institute of Control and Electronics Technology, Beijing, 100038, China

    Rufei Zhang, Nannan Li, Dongjin Li, Jianwei Lv & Jingyu Xu

Authors
  1. Yuchen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rufei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nannan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dongjin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianwei Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jingyu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yuchen Wang conceived the idea, designed and analyzed the experiment and wrote the main manuscript text. Wei Zhao guided the selection of topic and revised the final manuscript. Rufei Zhang and others managed and supervised the study progress. All authors reviewed the manuscript.

Corresponding author

Correspondence to Wei Zhao.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Zhao, W., Zhang, R. et al. Multi-object tracking in UAVs with feature fusion distribution and occlusion awareness. SIViP 19, 90 (2025). https://doi.org/10.1007/s11760-024-03715-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03715-y

Keywords