Skip to main content
SpringerLink
Log in

Super-Resolution-Assisted Feature Refined Extraction for Small Objects in Remote Sensing Images

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14555))

Included in the following conference series:

  • 312 Accesses

Abstract

Despite achieving impressive results in object detection in natural scenes, the task of object detection in remote sensing images is still full of challenges due to the large number of small objects in remote sensing images caused by the dense object distribution, complex backgrounds, and diverse scale variations. We propose a Super-Resolution-Assisted Feature Refined Extraction (SRRE) approach to address the difficulties of detecting small objects. Firstly, we employ a deeper level of feature fusion to effectively harness deep semantic information and shallow detailed information. Secondly, in the feature extraction process, a Feature Refined Extraction Module (FREM) is introduced to capture a wider range of contextual information, enhancing the global perceptual capability of features. Lastly, we introduce Super-Resolution (SR) branches at various feature layers to better integrate local textures and contextual information. We compared our method against commonly used approaches in remote sensing image object detection, including state-of-the-art (SOTA) methods. Our approach outperforms these methods and achieves superior results on the DOTA-v1.0, DIOR, and SODA-A datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cao, L., Zhang, X., Wang, Z., Ding, G.: Multi angle rotation object detection for remote sensing image based on modified feature pyramid networks. Int. J. Remote Sens. 42(14), 5253–5276 (2021). https://doi.org/10.1080/01431161.2021.1910371

    Article  Google Scholar 

  2. Cheng, G., et al.: Towards large-scale small object detection: survey and benchmarks. IEEE TPAMI (2023). https://doi.org/10.1109/tpami.2023.3290594

    Article  Google Scholar 

  3. Courtrai, L., Pham, M.T., Lefèvre, S.: Small object detection in remote sensing images based on super-resolution with auxiliary generative adversarial networks. Remote Sens. 12(19), 3152 (2020). https://doi.org/10.3390/rs12193152

    Article  Google Scholar 

  4. Ding, J., Xue, N., Long, Y., Xia, G.S., Lu, Q.: Learning roi transformer for oriented object detection in aerial images. In: ICCV, pp. 2849–2858 (2019). https://doi.org/10.1109/cvpr.2019.00296

  5. Han, J., Ding, J., Li, J., Xia, G.S.: Align deep features for oriented object detection. TGRS 60, 1–11 (2021). https://doi.org/10.1109/tgrs.2021.3062048

    Article  Google Scholar 

  6. Han, J., Ding, J., Xue, N., Xia, G.S.: Redet: a rotation-equivariant detector for aerial object detection. In: CVPR, pp. 2786–2795 (2021). https://doi.org/10.1109/cvpr46437.2021.00281

  7. Hong, M., Li, S., Yang, Y., Zhu, F., Zhao, Q., Lu, L.: Sspnet: scale selection pyramid network for tiny person detection from UAV images. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021). https://doi.org/10.1109/lgrs.2021.3103069

    Article  Google Scholar 

  8. Hou, L., Lu, K., Xue, J., Li, Y.: Shape-adaptive selection and measurement for oriented object detection. In: AAAI. vol. 36, pp. 923–932 (2022). https://doi.org/10.1609/aaai.v36i1.19975

  9. Li, K., Wan, G., Cheng, G., Meng, L., Han, J.: Object detection in optical remote sensing images: A survey and a new benchmark. ISPRS J. Photogramm. Remote. Sens. 159, 296–307 (2020). https://doi.org/10.1016/j.isprsjprs.2019.11.023

    Article  Google Scholar 

  10. Li, W., Chen, Y., Hu, K., Zhu, J.: Oriented reppoints for aerial object detection. In: CVPR, pp. 1829–1838 (2022). https://doi.org/10.1109/cvpr52688.2022.00187

  11. Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: CVPR Workshops, pp. 136–144 (2017). https://doi.org/10.1109/cvprw.2017.151

  12. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: CVPR, pp. 2117–2125 (2017). https://doi.org/10.1109/cvpr.2017.106

  13. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: ICCV, pp. 2980–2988 (2017). https://doi.org/10.1109/iccv.2017.324

  14. Lyu, C, et al.: Rtmdet: an empirical study of designing real-time object detectors. arXiv preprint arXiv:2212.07784 (2022). https://doi.org/10.48550/arXiv.2212.07784

  15. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016). https://doi.org/10.1109/cvpr.2016.91

  16. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015). https://doi.org/10.1109/tpami.2016.2577031

  17. Wang, C.Y., Liao, H.Y.M., Wu, Y.H., Chen, P.Y., Hsieh, J.W., Yeh, I.H.: Cspnet: a new backbone that can enhance learning capability of cnn. In: CVPR, pp. 390–391 (2020). https://doi.org/10.1109/cvprw50498.2020.00203

  18. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., Hu, Q.: Eca-net: efficient channel attention for deep convolutional neural networks. In: CVPR, pp. 11534–11542 (2020). https://doi.org/10.1109/cvpr42600.2020.01155

  19. Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: ECCV, pp. 3–19 (2018). https://doi.org/10.1007/978-3-030-01234-2_1

  20. Wu, J., Pan, Z., Lei, B., Hu, Y.: Fsanet: feature-and-spatial-aligned network for tiny object detection in remote sensing images. TGRS 60, 1–17 (2022). https://doi.org/10.1109/tgrs.2022.3205052

    Article  Google Scholar 

  21. Xia, G.S., et al.: Dota: a large-scale dataset for object detection in aerial images. In: CVPR, pp. 3974–3983 (2018). https://doi.org/10.1109/cvpr.2018.00418

  22. Xiaolin, F., et al.: Small object detection in remote sensing images based on super-resolution. Pattern Recogn. Lett. 153, 107–112 (2022). https://doi.org/10.3390/rs13091854

    Article  Google Scholar 

  23. Xie, X., Cheng, G., Wang, J., Yao, X., Han, J.: Oriented r-cnn for object detection. In: ICCV, pp. 3520–3529 (2021). https://doi.org/10.1109/iccv48922.2021.00350

  24. Xu, Y., et al.: Gliding vertex on the horizontal bounding box for multi-oriented object detection. TPAMI 43(4), 1452–1459 (2020). https://doi.org/10.1109/tpami.2020.2974745

    Article  Google Scholar 

  25. Yang, X., Yan, J., Feng, Z., He, T.: R3det: refined single-stage detector with feature refinement for rotating object. In: AAAI. vol. 35, pp. 3163–3171 (2021). https://doi.org/10.1609/aaai.v35i4.16426

  26. Yang, X., et al.: The kfiou loss for rotated object detection. arXiv preprint arXiv:2201.12558 (2022). https://doi.org/10.48550/arXiv.2201.12558

  27. Zhang, S., Chi, C., Yao, Y., Lei, Z., Li, S.Z.: Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In: CVPR, pp. 9759–9768 (2020). https://doi.org/10.1109/cvpr42600.2020.00978

  28. Zhang, W., Wang, S., Thachan, S., Chen, J., Qian, Y.: Deconv r-cnn for small object detection on remote sensing images. In: IGARSS, pp. 2483–2486. IEEE (2018). https://doi.org/10.1109/igarss.2018.8517436

  29. Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Trans. Comput. Imaging 3(1), 47–57 (2016). https://doi.org/10.1109/tci.2016.2644865

    Article  Google Scholar 

  30. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: CVPR, pp. 2921–2929 (2016). https://doi.org/10.1109/cvpr.2016.319

Download references

Acknowledgements

This work is supported by the project of the Engineering Research Center of Ecological Big Data, Ministry of Education, China).

Author information

Authors and Affiliations

  1. Inner Mongolia University, Hohhot, China

    Lihua Du & Wei Wu

  2. Dalian Maritime University, Dalian, China

    Chen Li

Authors
  1. Lihua Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Wu .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Stevan Rudinac

  2. Delft University of Technology, Delft, The Netherlands

    Alan Hanjalic

  3. Delft University of Technology, Delft, The Netherlands

    Cynthia Liem

  4. University of Amsterdam, Amsterdam, The Netherlands

    Marcel Worring

  5. Reykjavik University, Reykjavik, Iceland

    Björn Þór Jónsson

  6. Microsoft Research Lab – Asia, Beijing, China

    Bei Liu

  7. The University of Tokyo, Tokyo, Japan

    Yoko Yamakata

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Du, L., Wu, W., Li, C. (2024). Super-Resolution-Assisted Feature Refined Extraction for Small Objects in Remote Sensing Images. In: Rudinac, S., et al. MultiMedia Modeling. MMM 2024. Lecture Notes in Computer Science, vol 14555. Springer, Cham. https://doi.org/10.1007/978-3-031-53308-2_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53308-2_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53307-5

  • Online ISBN: 978-3-031-53308-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation