Skip to main content
SpringerLink
Log in

Arbitrary-oriented object detection via dense feature fusion and attention model for remote sensing super-resolution image

  • S.I. : Deep Learning Approaches for RealTime Image Super Resolution (DLRSR)
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, we aim at developing a new arbitrary-oriented end-to-end object detection method to further push the frontier of object detection for remote sensing image. The proposed method comprehensively takes into account multiple strategies, such as attention mechanism, feature fusion, rotation region proposal as well as super-resolution pre-processing simultaneously to boost the performance in terms of localization and classification under the faster RCNN-like framework. Specifically, a channel attention network is integrated for selectively enhancing useful features and suppressing useless ones. Next, a dense feature fusion network is designed based on multi-scale detection framework, which fuses multiple layers of features to improve the sensitivity to small objects. In addition, considering the objects for detection are often densely arranged and appear in various orientations, we design a rotation anchor strategy to reduce the redundant detection regions. Extensive experiments on two remote sensing public datasets DOTA, NWPU VHR-10 and scene text dataset ICDAR2015 demonstrate that the proposed method can be competitive with or even superior to the state-of-the-art ones, like R2CNN and R2CNN++.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR 2016), 2016, pp 770–778

  2. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition (CVPR), 2014, pp 580–587

  3. He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Proceedings of the 13th European conference on computer vision (ECCV 2014), 2014, pp 346–361

  4. Girshick R (2015) Fast R-CNN [region-based Convolutional Neural Network]. In: Proceedings of the 2015 IEEE international conference on computer vision (ICCV), 2015, pp 1440–1448

  5. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  6. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified real-time object detection. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), 2016, pp 779–788

  7. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. In: Proceedings of the 14th European conference computer vision (ECCV2016), 9905, pp 21–37

  8. Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. In: Proceedings of the 2016 conference on advances in neural information processing systems (NIPS), pp 379–387

  9. He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: Prcoeedings of the 2017 IEEE international conference on computer vision (ICCV), 2017, pp 2980–2988

  10. Yang X, Sun H, Kun F, Yang J, Sun X, Yan M, Guo Z (2018) Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sens 10(1):132–146

    Article  Google Scholar 

  11. Yang X, Sun H, Sun X, Yan M, Zhi G, Kun F (2018) Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network. IEEE Access 6:50839–50849

    Article  Google Scholar 

  12. Jiang Y, Zhu X, Wang X, Yang S, Li W, Wang H, Fu P, Luo Z, R2CNN: rotational region CNN for orientation robust scene text detection. arXiv:1706.09579

  13. Yang X, Fu K, Sun H, Yang J, Guo Z, Yan M, Zhang T, Xian S, R2CNN++: multi-dimensional attention based rotation invariant detector with robust anchor strategy. arXiv:1811.07126

  14. Ma J, Shao W, Ye H, Wang L, Wang H, Zheng Y, Xue X (2018) Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans Multimedia 20(11):3111–3122

    Article  Google Scholar 

  15. Shermeyer J, Van Etten A (2019) The effects of super-resolution on object detection performance in satellite imagery. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops

  16. Haris M, Shakhnarovich G, Ukita N (2018) Task-driven super resolution: object detection in low-resolution images. arXiv preprint arXiv:1803.11316

  17. Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. In: Proceedings of the 2017 conference on advances in neural information processing systems, 2017, pp 4468–4476

  18. Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, 936–944

  19. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation Networks. In: Proceedings of the 2018 IEEE conference on computer vision and pattern recognition, 2018, pp 7132–7141

  20. Uijlings JRR, van de Sande KEA, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171

    Article  Google Scholar 

  21. Shamsolmoali P, Zareapoor M, Wang R et al (2019) A novel deep structure U-net for sea-land segmentation in remote sensing images. IEEE J Sel Top Appl Earth Observ Remote Sens 12(9):3219–3232

    Article  Google Scholar 

  22. Shamsolmoali P, Zareapoor M, Wang R et al (2019) G-GANISR: gradual generative adversarial network for image super resolution. Neurocomputing 366:140–153

    Article  Google Scholar 

  23. Li F et al (2017) Super-resolution for GaoFen-4 remote sensing images. IEEE Geosci Remote Sens Lett 15(1):28–32

    Article  Google Scholar 

  24. Wu W et al (2016) A new framework for remote sensing image super-resolution: sparse representation-based method by processing dictionaries with multi-type features. J Syst Archit 64:63–75

    Article  Google Scholar 

  25. Xie S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp 5987–5995

  26. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp 2261–2269

  27. Xia G-S, Bai X, Ding J, Zhu Z, Belongie S, Luo J, Datcu M, Pelillo M, Zhang L (2018) DOTA: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) 2018, pp 3974–3983

  28. Cheng G, Zhou P, Han J (2016) Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Trans Geosci Remote Sens 54(12):7405–7415

    Article  Google Scholar 

  29. Zhang Y, Li K, Li K, Wang L, Zhong B, Yun F (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the IEEE conference on ECCV 2018, pp 1–16

  30. Eirikur A, Radu T (2017) NTIRE 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) 2017, pp 1–10

  31. Tang T, Zhou S, Deng Z, Lei L, Zou H (2017) Arbitrary oriented vehicle detection in aerial imagery with single convolutional neural networks. Remote Sens 9(11):1170

    Article  Google Scholar 

  32. Azimi SM, Vig E, Bahmanyar R, Körner M, Reinartz P (2018) Towards multi-class object detection in unconstrained remote sensing imagery. arXiv preprint, arXiv:1807.02700

  33. Zhou X et al (2017) EAST: an efficient and accurate scene text detector. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  34. Ren Y, Zhu C, Xiao S (2018) Deformable faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images. Remote Sens 10(9):1470

    Article  Google Scholar 

  35. Ding J et al (2018) Learning ROI transformer for detecting oriented objects in aerial images. arXiv preprint arXiv:1812.00155

  36. Han X, Zhong Y, Zhang L (2017) An efficient and robust integrated geospatial object detection framework for high spatial resolution remote sensing imagery. Remote Sens 9(7):666

    Article  Google Scholar 

  37. Liu W et al (2016) SSD: single shot multibox detector. In: European conference on computer vision. Springer, Cham

  38. Fu C-Y et al (2017) DSSD: deconvolutional single shot detector. arXiv preprint arXiv:1701.06659

  39. Shen Z et al (2017) DSOD: learning deeply supervised object detectors from scratch. In: Proceedings of the IEEE international conference on computer vision

  40. Xu Z et al (2017) Deformable convnet with aspect ratio constrained NMS for object detection in remote sensing imagery. Remote Sens 9(12):1312

    Article  Google Scholar 

  41. Li K et al (2017) Rotation-insensitive and context-augmented object detection in remote sensing images. IEEE Trans Geosci Remote Sens 56(4):2337–2348

    Article  Google Scholar 

  42. Tian Z et al (2016) Detecting text in natural image with connectionist text proposal network. In: European conference on computer vision. Springer, Cham

  43. Shi B, Bai X, Belongie S (2017) Detecting oriented text in natural images by linking segments. In: Proceedings of the IEEE conference on computer vision and pattern recognition

Download references

Acknowledgements

This work is supported in part by the National Natural Science Foundation of China under Grant Nos. 61672254, 61672246, 61572221 and 61300222, Program for Hust Academic Frontier Youth Team, Key project of National Natural Science Foundation of China Grant No. U1536203, Natural Science Foundation of Hubei Province Grant No. 2015CFB687 and the Fundamental Research Funds for the Central Universities, HUST: 2016YXMS088 and 2016YXMS018. The authors appreciate the valuable suggestions from the anonymous reviewers and the Editors.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Fuhao Zou, Wei Xiao, Kunkun He & Kai Li

  2. School of Natural and Computational Science, Massey University, Auckland, New Zealand

    Wanting Ji

  3. Wuhan Digital Engineering Research Institute, Wuhan, China

    Zhixiang Yang

  4. Innovation Center, University of Electronic Science and Technology of China, Chengdu, China

    Jingkuan Song

  5. School of Engineering, Manukau Institute of Technology, Auckland, New Zealand

    Helen Zhou

Authors
  1. Fuhao Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wanting Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kunkun He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhixiang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jingkuan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Helen Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuhao Zou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, F., Xiao, W., Ji, W. et al. Arbitrary-oriented object detection via dense feature fusion and attention model for remote sensing super-resolution image. Neural Comput & Applic 32, 14549–14562 (2020). https://doi.org/10.1007/s00521-020-04893-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-04893-9

Keywords

Search

Navigation