Skip to main content
SpringerLink
Log in

Adaptive Kalman Filter with power transformation for online multi-object tracking

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

By introducing a low-score detection box association stage, the full-detection association can effectively enhance the accuracy and robustness of online multi-object tracking. However, this association would lead to a decline in tracking precision, the key point of which is that the fixed noise setting of Kalman Filter is difficult to balance the system requirements for high-score and low-score detection boxes. A Power-Adaptive Kalman Filter (PAKF) was proposed in this article. Taking the motion matching cost and confidence score as process and observation noise scale parameters, respectively, and combined with the power transformation, two adaptive factors were constructed to adjust the process and observation covariance matrices, respectively. Sufficient ablation experiments were conducted on the full validation set of MOT17. After introducing the PAKF into the ByteTrack and SORT, the High-Order Tracking Accuracy, Multi-Object Tracking Precision (MOTP) and ID F1 score of them were improved by about 1%, and their improvements were more obvious in complex scenarios. On the challenging HiEve benchmark dataset, after introducing the PAKF, the Multi-Object Tracking Accuracy and MOTP of the ByteTrack were improved by 0.53% and 0.28%, respectively. It is more advantageous than other state-of-the-art online methods. The proposed PAKF can effectively improve the performances of the multi-object tracking algorithms based on the Kalman Filter and tracking-by-detection. The codes are available at https://github.com/LiYi199983/PAKF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The authors confirm that the code, network weights and datasets supporting the results of this study can be found in the article.

References

  1. Kishi, N., Shinkuma, R., Oka, M., et al.: Multi-object tracking for road surveillance without using features of image data. In: 2021 IEEE Global Communications Conference (GLOBECOM). pp. 1–6 (2021).

  2. Yurtsever, M.M.E., Eken, S.: BabyPose: real-time decoding of baby’s non-verbal communication using 2D video-based pose estimation. IEEE Sens. J. 22(14), 13776–13784 (2022). https://doi.org/10.1109/jsen.2022.3183502

    Article  Google Scholar 

  3. Chen, J., Wang, F., Li, C., et al.: Online multiple object tracking using a novel discriminative module for autonomous driving. Electronics 10(20), 2479 (2021). https://doi.org/10.3390/electronics10202479

    Article  Google Scholar 

  4. Ge, Z., Liu, S., Wang, F., et al.: YOLOX: Exceeding YOLO Series in 2021. (2021). https://ui.adsabs.harvard.edu/abs/2021arXiv210708430G

  5. Zhou, X., Wang, D., Krähenbühl, P. J. a. e.-p.: Objects as Points. (2019). https://ui.adsabs.harvard.edu/abs/2019arXiv190407850Z

  6. Ahmed, I., Ahmad, M., Ahmad, A., et al.: Top view multiple people tracking by detection using deep SORT and YOLOv3 with transfer learning: within 5G infrastructure. Int. J. Mach. Learn. Cybern. 12(11), 3053–3067 (2021)

    Article  Google Scholar 

  7. Guo, W., Jin, Y., Shan, B., et al.: Multi-cue multi-hypothesis tracking with re-identification for multi-object tracking. Multimed. Syst. 28(3), 925–937 (2022)

    Article  Google Scholar 

  8. Sheng, H., Chen, J., Zhang, Y., et al.: Iterative multiple hypothesis tracking with tracklet-level association. IEEE Trans. Cir. Syst. Video Technol. 29(12), 3660–3672 (2018)

    Article  Google Scholar 

  9. Song, Y.-m., Jeon, M.: Online multiple object tracking with the hierarchically adopted gm-phd filter using motion and appearance. In: 2016 IEEE International conference on consumer electronics-Asia (ICCE-Asia). pp. 1–4 (2016)

  10. Wang, Z., Zheng, L., Liu, Y., et al.: Towards real-time multi-object tracking. In: European Conference on Computer Vision. pp. 107–122 (2020)

  11. Zhang, Y., Wang, C., Wang, X., et al.: Fairmot: on the fairness of detection and re-identification in multiple object tracking. Int. J. Comput. Vision 129(11), 3069–3087 (2021)

    Article  MathSciNet  Google Scholar 

  12. Pang, J., Qiu, L., Li, X., et al.: Quasi-dense similarity learning for multiple object tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 164–173 (2021).

  13. Zhou, X., Koltun, V., Krähenbühl, P.: Tracking objects as points. In: European Conference on Computer Vision. pp. 474–490 (2020).

  14. Hua, W., Mu, D., Zheng, Z., et al.: Online multi-person tracking assist by high-performance detection. J. Supercomput. 76(6), 4076–4094 (2017). https://doi.org/10.1007/s11227-017-2202-8

    Article  Google Scholar 

  15. Chen, L., Ai, H., Zhuang, Z., et al.: Real-time multiple people tracking with deeply learned candidate selection and person re-identification. In: 2018 IEEE International Conference on Multimedia and Expo (ICME). pp. 1–6 (2018).

  16. Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. In: 2017 IEEE International Conference on Image Processing (ICIP). pp. 3645–3649 (2017).

  17. Min, Y., Yuwei, W., Yunde, J.: A hybrid data association framework for robust online multi-object tracking. IEEE Trans. Image Process. 26(12), 5667–5679 (2017). https://doi.org/10.1109/TIP.2017.2745103

    Article  MathSciNet  MATH  Google Scholar 

  18. Bae, S.H., Yoon, K.J.: Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 595–610 (2018). https://doi.org/10.1109/TPAMI.2017.2691769

    Article  Google Scholar 

  19. Zhang, Y., Sun, P., Jiang, Y., et al.: ByteTrack: Multi-Object Tracking by Associating Every Detection Box (2021). https://ui.adsabs.harvard.edu/abs/2021arXiv211006864Z

  20. Kalman, R.E.: A new approach to linear filtering and prediction problems. J Fluids Eng. 82(1), 35–45 (1960)

    MathSciNet  Google Scholar 

  21. Bewley, A., Ge, Z., Ott, L., et al.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing (ICIP). pp. 3464–3468 (2016).

  22. Hou, X., Wang, Y., Chau, L.-P.: Vehicle tracking using deep sort with low confidence track filtering. In: 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). pp. 1–6 (2019).

  23. He, J., Erfani, S., Ma, X., et al.: Alpha-IoU: a family of power intersection over union losses for bounding box regression. Adv. Neural. Inf. Process. Syst. 34, 20230–20242 (2021)

    Google Scholar 

  24. Du, Y., Wan, J., Zhao, Y., et al.: GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2809–2819 (2021).

  25. Sarkar, K., Halder, T.K., Mandal, A.: Adaptive power-law and cdf based geometric transformation for low contrast image enhancement. Multimed. Tools Appl. 80(4), 6329–6353 (2021)

    Article  Google Scholar 

  26. Milan, A., Leal-Taixe, L., Reid, I., et al.: MOT16: A Benchmark for Multi-Object Tracking (2016). https://ui.adsabs.harvard.edu/abs/2016arXiv160300831M

  27. Lin, W., Liu, H., Liu, S., et al.: Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events (2020). https://ui.adsabs.harvard.edu/abs/2020arXiv200504490L

  28. Lu, Z., Rathod, V., Votel, R., et al.: Retinatrack: Online single stage joint detection and tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14668–14678 (2020).

  29. Sener, O., Koltun, V.: Multi-task learning as multi-objective optimization. In: the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018). (2018).

  30. Zhang, J., Xing, M., Sun, G.-C., et al.: Multiple statistics contributing to few-sample deep learning for subtle trace detection in high-resolution SAR images. IEEE Trans. Geosci. Remote Sens. 60, 1–14 (2021)

    Google Scholar 

  31. Kim, T., Park, T.-H.: Extended Kalman filter (EKF) design for vehicle position tracking using reliability function of radar and lidar. Sensors. 20(15), 4126 (2020)

    Article  Google Scholar 

  32. Wang, M., Wu, X.: Multi-object tracking strategy of autonomous vehicle using modified unscented Kalman Filter and reference point switching. J. Shanghai Jiaotong University (Science). 26(5), 607–614 (2021)

    Article  Google Scholar 

  33. Cui, Y., Zhang, J., He, Z., et al.: Multiple pedestrian tracking by combining particle filter and network flow model. Neurocomputing 351, 217–227 (2019)

    Article  Google Scholar 

  34. Guo, W., Zhao, Q., Gu, D.: Visual tracking using an insect vision embedded particle filter. Math. Prob. Eng. (2015). https://doi.org/10.1155/2015/573131

    Article  Google Scholar 

  35. Khattak, A.S., Raja, G., Anjum, N.: Adaptive framework for multi-feature hybrid object tracking. Appl. Sci. 8(11), 2294 (2018)

    Article  Google Scholar 

  36. Zhai, G., Wu, C., Wang, Y.: Millimeter wave radar target tracking based on adaptive Kalman filter. In: 2018 IEEE Intelligent Vehicles Symposium (IV). pp. 453–458 (2018).

  37. Wang, Y., Mu, X.: Dynamic siamese network with adaptive Kalman filter for object tracking in complex scenes. IEEE Access. 8, 222918–222930 (2020)

    Article  Google Scholar 

  38. Cao, J., Weng, X., Khirodkar, R., et al.: Observation-centric SORT: rethinking SORT for robust multi-object tracking. (2022). https://ui.adsabs.harvard.edu/abs/2022arXiv220314360C

  39. Kumar, M., Singh, K.: Retrieval of head–neck medical images using Gabor filter based on power-law transformation method and rank BHMT. Signal Image Video Process. 12(5), 827–833 (2018)

    Article  Google Scholar 

  40. Lin, T.-Y., Goyal, P., Girshick, R., et al.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2980–2988 (2017).

  41. Dendorfer, P., Rezatofighi, H., Milan, A., et al.: MOT20: a benchmark for multi object tracking in crowded scenes. (2020). https://ui.adsabs.harvard.edu/abs/2020arXiv200309003D

  42. Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. EURASIP J. Image Video Process. 2008, 1–10 (2008)

    Article  Google Scholar 

  43. Ristani, E., Solera, F., Zou, R., et al.: Performance measures and a data set for multi-target, multi-camera tracking. In: European conference on computer vision. pp. 17–35 (2016).

  44. Luiten, J., Os Ep, A.A., Dendorfer, P., et al.: HOTA: a higher order metric for evaluating multi-object tracking. Int. J. Comput. Vis. 129(2), 548–578 (2021). https://doi.org/10.1007/s11263-020-01375-2

    Article  Google Scholar 

  45. Wu, B., Nevatia, R.: Tracking of multiple, partially occluded humans based on static body part detection. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06). pp. 951–958 (2006).

  46. Shao, S., Zhao, Z., Li, B., et al.: Crowdhuman: a benchmark for detecting human in a crowd. (2018). https://doi.org/10.48550/arXiv.1805.00123

  47. Zhang, S., Benenson, R., Schiele, B.: Citypersons: a diverse dataset for pedestrian detection. In: Proceedings of the IEEE Conference on Computer VISION and Pattern Recognition. pp. 3213–3221 (2017)

  48. Ess, A., Leibe, B., Schindler, K., et al.: A mobile vision system for robust multi-person tracking. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition. pp. 1–8 (2008)

  49. Shuai, B., Berneshawi, A., Wang, M., et al.: Application of multi-object tracking with Siamese track-RCNN to the human in events dataset. In: Proceedings of the 28th ACM International Conference on Multimedia. pp. 4625–4629 (2020)

Download references

Acknowledgements

This work was supported in part by Anhui Province Key Laboratory of Advanced Numerical Control & Servo Technology, Grant (No. XJSK202102), by the Special Fund for Collaborative Innovation of Anhui Polytechnic University & Jiujiang District under Grant (No. 2021CYXTB3), and by the Natural Science Research Project of Higher Education of Anhui Province of China under Grant (No. YJS20210446).

Author information

Authors and Affiliations

  1. The Mechanical Engineering Department, Anhui Polytechnic University, Wuhu, 241000, China

    Youyu Liu, Yi Li, Dezhang Xu & Wanbao Tao

  2. The College of Mechanical and Electrical Engineering, Anhui Jianzhu University, Hefei, 230601, China

    Qingyan Yang

Authors
  1. Youyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dezhang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingyan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wanbao Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization and methodology, Youyu Liu and Yi Li; software, Yi Li; validation, Yi Li and Qingyan Yang; formal analysis, Dezhang Xu; writing—original draft preparation, Youyu Liu; writing—review and editing, Youyu Liu; visualization, Yi Li; project administration, Youyu Liu and Wanbao Tao; funding acquisition, Youyu Liu and Qingyan Yang. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Qingyan Yang.

Ethics declarations

Conflict of interest

This paper does not contain any studies with human or animal subjects and all authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Li, Y., Xu, D. et al. Adaptive Kalman Filter with power transformation for online multi-object tracking. Multimedia Systems 29, 1231–1244 (2023). https://doi.org/10.1007/s00530-023-01052-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-023-01052-7

Keywords

Search

Navigation