Skip to main content
SpringerLink
Log in

A unified and costless approach for improving small and long-tail object detection in aerial images of traffic scenarios

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Object detection is an important and challenging task for the utilization of arterial images. However, in traffic scenarios, small-sized and long-tail distributed object detections are still major challenges for practical applications. Most previous studies consider these two important problems as irrelevant issues and only tackle one of them at a time. In addition, most of these works achieve improvements at the cost of significant computation increase. In this article, we unveil that both object size and frequency should be taken into consideration in a unified manner. Then, we intend to simultaneously improve small-sized and long-tail distributed object detection at training stage. In specific, (1) we first propose a scale-aware label assignment strategy to focus on relatively small objects, it intuitively adjusts sampling areas according to the size of target. (2) Based on the assignment results, a dynamic threshold is generated and subsequently integrated into a dynamic class suppression loss, which suppresses unnecessary negative gradient to boost the performance on tail categories from a statistic-free perspective. We conduct extensive experiments on typical datasets, i.e., VisDrone and UAVDT, results show that the proposed method achieves 6.5% and 7.6% improvements over the baseline, which sets a new state of the art. We further test the proposed approach for real-time detection on an edge device and also show effectiveness of the proposed method. Without aggravating time and computation burden, our method effectively enhances the performance of object detection in aerial images of traffic scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: Proceedings of the european conference on computer vision, pp 740–755. Springer

  2. Cao Y, He Z, Wang L, Wang W, Yuan Y, Zhang D, Zhang J, Zhu P, Van Gool L, Han J et al (2021) Visdrone-det2021: The vision meets drone object detection challenge results. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 2847–2854

  3. Xiong Z, Li M, Ma Y, Wu X (2020) Vehicle re-identification with image processing and car-following model using multiple surveillance cameras from urban arterials. IEEE Trans Intell Transp Syst 22 (12):7619–7630

    Article  Google Scholar 

  4. Wei Z, Duan C, Song X, Tian Y, Wang H (2020) Amrnet:, Chips augmentation in aerial images object detection. arXiv:2009.07168

  5. Yu W, Yang T, Chen C (2021) Towards resolving the challenge of long-tail distribution in uav images for object detection. In: Proceedings of the IEEE/CVF Winter conference on applications of computer vision, pp 3258–3267

  6. Yang C, Huang Z, Wang N (2022) Querydet: Cascaded sparse query for accelerating high-resolution small object detection. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 1–11

  7. Sun W, Dai L, Zhang X, Chang P, He X (2021) Rsod: Real-time small object detection algorithm in uav-based traffic monitoring. Applied Intelligence, pp 1–16

  8. Yang F, Fan H, Chu P, Blasch E, Ling H (2019) Clustered object detection in aerial images. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 8311–8320

  9. Li C, Yang T, Zhu S, Chen C, Guan S (2020) Density map guided object detection in aerial images. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition workshops, pp 190–191

  10. He K, Gkioxari G, Dollaŕ P, Girshick R (2020) Mask r-cnn. IEEE Trans Pattern Anal Mach Intell 42(2):386–397

    Article  Google Scholar 

  11. Cai Z, Vasconcelos N (2019) Cascade r-cnn: high quality object detection and instance segmentation. IEEE Trans Pattern Anal Mach Intell 43(5):1483–1498

    Article  Google Scholar 

  12. Zhang S, Wen L, Lei Z, Li SZ (2020) Refinedet++: Single-shot refinement neural network for object detection. IEEE Trans Circuits Syst Video Technol 31(2):674–687

    Article  Google Scholar 

  13. Choi J, Chun D, Kim H, Lee H-J (2019) Gaussian yolov3: an accurate and fast object detector using localization uncertainty for autonomous driving. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 502–511

  14. Cai Y, Luan T, Gao H, Wang H, Chen L, Li Y, Sotelo MA, Li Z (2021) Yolov4-5d: an effective and efficient object detector for autonomous driving. IEEE Trans Instrum Meas 70:1–13

    Google Scholar 

  15. Tian Z, Shen C, Chen H, He T (2022) Fcos: a simple and strong anchor-free object detector. IEEE Trans Pattern Anal Mach Intell 44(4):1922–1933

    Google Scholar 

  16. Zhou X, Wang D, Krähenbühl P (2019) Objects as points. arXiv:1904.07850

  17. Law H, Deng J (2020) Cornernet: Detecting objects as paired keypoints. Int J Comput Vis 128(3):642–656

    Article  Google Scholar 

  18. Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q (2019) Centernet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 6569–6578

  19. Ge Z, Liu S, Li Z, Yoshie O, Sun J (2021) Ota: Optimal transport assignment for object detection. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 303–312

  20. Ge Z, Liu S, Wang F, Li Z, Sun J (2021) Yolox:, Exceeding yolo series in 2021. arXiv:2107.08430

  21. Cui Y, Jia M, Lin T-Y, Song Y, Belongie S (2019) Class-balanced loss based on effective number of samples. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 9268–9277

  22. Du J, Liu L, Li R, Jiao L, Xie C, Wang R (2022) Towards densely clustered tiny pest detection in the wild environment. Neurocomputing 490:400–412

    Article  Google Scholar 

  23. Chen G, Wang H, Chen K, Li Z, Song Z, Liu Y, Chen W, Knoll A (2020) A survey of the four pillars for small object detection: Multiscale representation, contextual information, super-resolution, and region proposal. IEEE Transactions on systems, man and cybernetics: systems

  24. Tong K, Wu Y, Zhou F (2020) Recent advances in small object detection based on deep learning: a review. Image Vis Comput 97:103910

    Article  Google Scholar 

  25. Bai Y, Zhang Y, Ding M, Ghanem B (2018) Sod-mtgan: Small object detection via multi-task generative adversarial network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 206–221

  26. Tan M, Pang R, Le QV (2020) Efficientdet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 10781–10790

  27. Chalavadi V, Jeripothula P, Datla R, Ch SB et al (2022) msodanet: a network for multi-scale object detection in aerial images using hierarchical dilated convolutions. Pattern Recogn 126:108548

    Article  Google Scholar 

  28. Mittal P, Sharma A, Singh R, Dhull V (2022) Dilated convolution based rcnn using feature fusion for low-altitude aerial objects. Expert Syst Appl 199:117106

    Article  Google Scholar 

  29. Chen Y, Zheng W, Zhao Y, Song TH, Shin H (2022) Dw-yolo: an efficient object detector for drones and self-driving vehicles. Arab J Sci Eng, pp 1–10

  30. Zhang S, Chi C, Yao Y, Lei Z, Li SZ (2020) Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 9759–9768

  31. Kim K, Lee HS (2020) Probabilistic anchor assignment with iou prediction for object detection. In: Proceedings of the european conference on computer vision, pp 355–371. Springer

  32. Wan X, Yu J, Tan H, Wang J (2022) Lag: Layered objects to generate better anchors for object detection in aerial images. Sensors 22(10):3891

    Article  Google Scholar 

  33. Oksuz K, Cam BC, Kalkan S, Akbas E (2020) Imbalance problems in object detection: a review. IEEE Trans Pattern Anal Mach Intell 43(10):3388–3415

    Article  Google Scholar 

  34. Wang T, Li Y, Kang B, Li J, Liew J, Tang S, Hoi S, Feng J (2020) The devil is in classification: A simple framework for long-tail instance segmentation. In: Proceedings of the european conference on computer vision, pp 728–744. Springer

  35. Chang N, Yu Z, Wang Y-X, Anandkumar A, Fidler S, Alvarez JM (2021) Image-level or object-level? a tale of two resampling strategies for long-tailed detection. In: International conference on machine learning, pp 1463–1472

  36. Tan J, Wang C, Li B, Li Q, Ouyang W, Yin C, Yan J (2020) Equalization loss for long-tailed object recognition. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 11662–11671

  37. Tan J, Lu X, Zhang G, Yin C, Li Q (2021) Equalization loss v2: a new gradient balance approach for long-tailed object detection. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 1685–1694

  38. Li Y, Wang T, Kang B, Tang S, Wang C, Li J, Feng J (2020) Overcoming classifier imbalance for long-tail object detection with balanced group softmax. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 10991–11000

  39. Wang J, Zhang W, Zang Y, Cao Y, Pang J, Gong T, Chen K, Liu Z, Loy CC, Lin D (2021) Seesaw loss for long-tailed instance segmentation. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 9695–9704

  40. Feng C, Zhong Y, Huang W (2021) Exploring classification equilibrium in long-tailed object detection. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 3417–3426

  41. Li B, Yao Y, Tan J, Zhang G, Yu F, Lu J, Luo Y (2022) Equalized focal loss for dense long-tailed object detection. arXiv:2201.02593

  42. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE International conference on computer vision, pp 2980–2988

  43. Wang T, Zhu Y, Zhao C, Zeng W, Wang J, Tang M (2021) Adaptive class suppression loss for long-tail object detection. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 3103–3112

  44. Chen X, Li Z, Jiang J, Han Z, Deng S, Li Z, Fang T, Huo H, Li Q, Liu M (2020) Adaptive effective receptive field convolution for semantic segmentation of vhr remote sensing images. IEEE Trans GeoSci Remote Sens 59(4):3532–3546

    Article  Google Scholar 

  45. Yu H, Li G, Zhang W, Huang Q, Du D, Tian Q, Sebe N (2020) The unmanned aerial vehicle benchmark: Object detection, tracking and baseline. Int J Comput Vis 128(5):1141–1159

    Article  Google Scholar 

  46. Megvii-BaseDetection (2021) https://github.com/Megvii-BaseDetection.https://github.com/Megvii-BaseDetection. Accessed 1 Oct 2022

  47. Yu G, Chang Q, Lv W, Xu C, Cui C, Ji W, Dang Q, Deng K, Wang G, Du Y et al (2021) Pp-picodet:, A better real-time object detector on mobile devices. arXiv:2111.00902

Download references

Acknowledgements

We would also like to thank the insightful and constructive comments from anonymous reviewers.

Funding

This work was partially supported by the National Science Foundation of China (52172376), the Young Scientists Fund of the National Natural Science Foundation of China (52002013) and the China Postdoctoral Science Foundation (BX20200036, 2020M680298).

Author information

Authors and Affiliations

  1. School of Transportation Science and Engineering, Beihang University, Beijing, 100191, China

    Zhongxia Xiong, Tao Song, Shan He, Ziying Yao & Xinkai Wu

  2. Beijing Key Laboratory for Cooperative Vehicle Infrastructure Systems and Safety Control, Beihang University, Beijing, 100191, China

    Zhongxia Xiong, Tao Song, Shan He, Ziying Yao & Xinkai Wu

Authors
  1. Zhongxia Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shan He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ziying Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xinkai Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Zhongxia Xiong designed the overall approach and wrote the paper. Xinkai Wu designed the experiments and revised the article. Tao Song and Shan He performed and analyzed the experiment results. Ziying Yao made contribution on data collection.

Corresponding author

Correspondence to Xinkai Wu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiong, Z., Song, T., He, S. et al. A unified and costless approach for improving small and long-tail object detection in aerial images of traffic scenarios. Appl Intell 53, 14426–14447 (2023). https://doi.org/10.1007/s10489-022-04108-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04108-9

Keywords

Search

Navigation