Skip to main content
SpringerLink
Log in

A two-way dense feature pyramid networks for object detection of remote sensing images

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

The bird’s eye view, multi-scale and dense classes in remote sensing images challenge the object detection of remote sensing images. It is not satisfactory to directly apply the object detection method designed for natural scene images to the object detection of remote sensing images. In this paper, we propose a detector with enhanced feature extraction ability to solve the above challenges, namely TWDFPN. TWDFPN has designed the structure of a two-way feature pyramid network (TWFPN) by combining feature maps with different generation directions and different spatial resolutions, which not only improves the utilization of the underlying feature information, but also strengthens the repeated utilization of the feature information of the backbone network, and ultimately improves the feature extraction ability of the network. Meanwhile, the dense-connected module is used in TWFPN to enhance the feature representation ability through limited additional computation cost, which extends the network and deepens the network. To evaluate the effectiveness of the proposed algorithm, this paper carried out experiments on NWPUVHR-10 and RSOD public remote sensing datasets, and the average accuracy (mAP) of 92.98% and 96.16%, respectively, which achieves advanced performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig.10
Fig.11

Similar content being viewed by others

References

  1. Shivappriya SN, Priyadarsini MJP, Stateczny A et al (2021) Cascade object detection and remote sensing object detection method based on trainable activation function. Remote Sens 13(2):200

    Article  Google Scholar 

  2. Li K, Wan G, Cheng G et al (2020) Object detection in optical remote sensing images: a survey and a new benchmark. ISPRS J Photogramm Remote Sens 159:296–307

    Article  Google Scholar 

  3. Jiao L, Zhang F, Liu F et al (2019) A survey of deep learning-based object detection. IEEE Access 7:128837–128868

    Article  Google Scholar 

  4. Sun P, Zheng Y, Zhou Z et al (2020) R4 Det: refined single-stage detector with feature recursion and refinement for rotating object detection in aerial images. Image Vis Comput 103:104036

    Article  Google Scholar 

  5. Zou Z, Chen K, Shi Z, et al. (2023) Object detection in 20 years: a survey. In: Proceedings of the IEEE

  6. Zhang M, Chen Y, Liu X et al (2020) Adaptive anchor networks for multi-scale object detection in remote sensing images. IEEE Access 8:57552–57565

    Article  Google Scholar 

  7. Liu L, Ouyang W, Wang X et al (2020) Deep learning for generic object detection: a survey. Int J Comput Vis 128:261–318

    Article  MATH  Google Scholar 

  8. Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv:2004.10934

  9. Li Z, Wang Y, Zhang N et al (2022) Deep learning-based object detection techniques for remote sensing images: a survey. Remote Sens 14(10):2385

    Article  Google Scholar 

  10. Lin TY, Doll´ar P, Girshick R, et al. (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision pattern recognition, 2117–2125

  11. Liu S, Qi L, Qin H, et al. (2018) Path aggregation network for instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 8759–8768

  12. Tan M, Pang R, Le QV (2020) Efficientdet: scalable and efficient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10781–10790

  13. Ghaisi G, Lin TY, Pang R, et al. Learning scalable feature pyramid architecture for object detection. In: Proceedings of the IEEE computer vision and pattern recognition, 7029–7038

  14. Zhang X, Zhu K, Chen G et al (2019) Geospatial object detection on high resolution remote sensing imagery based on double multi-scale feature pyramid network. Remote Sens 11(7):755

    Article  Google Scholar 

  15. Fu K, Chen Z, Zhang Y et al (2019) Enhanced feature representation in detection for optical remote sensing images. Remote Sens 11(18):2095

    Article  Google Scholar 

  16. Wang C, Bai X, Wang S et al (2018) Multiscale visual attention networks for object detection in VHR remote sensing images. IEEE Geosci Remote Sens Lett 16(2):310–314

    Article  Google Scholar 

  17. Qu J, Su C, Zhang Z et al (2020) Dilated convolution and feature fusion SSD network for small object detection in remote sensing images. IEEE Access 8:82832–82843

    Article  Google Scholar 

  18. Su H, Wei S, Liu S et al (2020) HQ-ISNet: high-quality instance segmentation for remote sensing imagery. Remote Sens 12(6):989

    Article  Google Scholar 

  19. Lin Y, He H, Yin Z et al (2014) Rotation-invariant object detection in remote sensing images based on radial-gradient angle. IEEE Geosci Remote Sens Lett 12(4):746–750

    Google Scholar 

  20. Feng C, Cao Z, Xiao Y et al (2023) Multi-spectral template matching based object detection in a few-shot learning manner. Inf Sci 624:20–36

    Article  Google Scholar 

  21. Ok AO (2013) Automated detection of buildings from single VHR multispectral images using shadow information and graph cuts. ISPRS J Photogramm Remote Sens 86:21–40

    Article  Google Scholar 

  22. Lin Z, Zhu F, Kong Y et al (2022) SRSG and S2SG: a model and a dataset for scene graph generation of remote sensing images from segmentation results. IEEE Trans Geosci Remote Sens 60:1–11

    Google Scholar 

  23. Blaschke T (2010) Object based image analysis for remote sensing. ISPRS J Photogramm Remote Sens 65(1):2–16

    Article  Google Scholar 

  24. Garajeh MK, Feizizadeh B, Blaschke T et al (2022) Detecting and mapping karst landforms using object-based image analysis: case study: Takht-Soleiman and Parava Mountains, Iran. Egypt J Remote Sens Space Sci 25(2):473–489

    Google Scholar 

  25. Li Y, Wang S, Tian Q et al (2015) Feature representation for statistical- learning-based object detection: a review. Pattern Recognit 48(11):3542–3559

    Article  Google Scholar 

  26. Mahadevkar SV, Khemani B, Patil S, et al. (2022) A review on machine learning styles in computer vision-techniques and future directions. IEEE Access

  27. Girshick R, Donahue J, Darrell T, et al. (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 580–587

  28. Ren S, He K, Girshick R, et al. (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28

  29. Lin TY, Goyal P, Girshick R, et al. (2017) Focal loss for dense object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2980–2988

  30. He K, Gkioxari G, Doll´ar P, et al. (2017) Mask r-cnn. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2961–2969

  31. Redmon J, Divvala S, Girshick R, et al. (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 779–788

  32. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv:1804.02767

  33. Liu W, Anguelov D, Erhan D et al (2016) Ssd: single shot multibox detector. Springer, Berlin, pp 21–37

    Google Scholar 

  34. Zhang F, Du B, Zhang L et al (2016) Weakly supervised learning based on coupled convolutional neural networks for aircraft detection. IEEE Trans Geosci Remote Sens 54(9):5553–5563

    Article  Google Scholar 

  35. Pang J, Li C, Shi J, et al. (2019) R2-CNN: fast tiny object detection in large- scale remote sensing images. arXiv:1902.06042

  36. Li Y, Huang Q, Pei X et al (2020) RADet: refine feature pyramid network and multi-layer attention network for arbitrary-oriented object detection of remote sensing images. Remote Sens 12(3):389

    Article  Google Scholar 

  37. Li C, Cong R, Guo C et al (2020) A parallel down-up fusion network for salient object detection in optical remote sensing images. Neurocomputing 415:411–420

    Article  Google Scholar 

  38. Qiao S, Chen LC, Yuille A (2021) Detectors: detecting objects with recursive feature pyramid and switchable atrous convolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10213–10224

  39. He K, Zhang X, Ren S, et al. (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778

  40. Huang G, Liu Z, Van Der Maaten L, et al. (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 4700–4708

  41. Cheng G, Han J, Zhou P et al (2014) Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS J Photogramm Remote Sens 98:119–132

    Article  Google Scholar 

  42. Xiao Z, Liu Q, Tang G et al (2015) Elliptic Fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images. Int J Remote Sens 36(2):618–644

    Article  Google Scholar 

  43. Ge Z, Liu S, Wang F, et al. (2021) Yolox: exceeding yolo series in 2021. arXiv:2107.08430

  44. Zhang K, Shen H (2022) Multi-stage feature enhancement pyramid network for detecting objects in optical remote sensing images. Remote Sens 14(3):579

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the Heilongjiang Provincial Natural Science Foundation (LH2022F047) and the Special Fund of Fundamental Scientific Research Business Expense for Higher School of Heilongjiang Province (2021-KYYWF-0002).

Author information

Authors and Affiliations

  1. College of Electronic Engineering, Heilongjiang University, Harbin, 150080, China

    Haocong Li, Hui Ma, Yanbo Che & Zedong Yang

Authors
  1. Haocong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanbo Che
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zedong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HL contributed to methodology, software, investigation, writing—original draft, and data Curation. HM contributed to conceptualization, writing—review and editing, supervision, and data Curation. YC contributed to software ZY contributed to resources. All authors reviewed the manuscript.

Corresponding author

Correspondence to Hui Ma.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Ma, H., Che, Y. et al. A two-way dense feature pyramid networks for object detection of remote sensing images. Knowl Inf Syst 65, 4847–4871 (2023). https://doi.org/10.1007/s10115-023-01916-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-01916-4

Keywords

Search

Navigation