Skip to main content
SpringerLink
Log in

ROPDet: real-time anchor-free detector based on point set representation for rotating object

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Remote-sensing object detection is a challenging task due to the difficulties of separating the objects with arbitrary direction from complex backgrounds. Though substantial progress has been made, there still exist challenges for object detection under the scenario of small scale, large aspect ratio, and dense distribution. Besides, the current mainstream approach falls under anchor-based multi-stage method, which has a serious shortcoming of slower inference speed. To conquer the aforementioned issues, this paper used RoPoints (points in rotation objects), a new better representation of objects as a set of sample points to perform object localization and classification. Then, we propose an anchor-free refined rotation detector:ROPDet based on RoPoints for more accurate and faster object detection. In our method, there is no need to predefine a large number anchors with different shapes. We only need to learn RoPoints for each object followed by converting to the corresponding bounding box, which greatly accelerates the inference process. Extensive experiments on two public remote-sensing datasets DOTA and HRSC-2016 demonstrate the competitive ability in terms of accuracy and inference speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Azimi, S.M., Vig, E., Bahmanyar, R., Körner, M., Reinartz, P.: Towards multi-class object detection in unconstrained remote sensing imagery. In: Asian Conference on Computer Vision. Springer, pp. 150–165 (2018)

  2. Cai, Z., Vasconcelos, N.: Cascade r-cnn: delving into high quality object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6154–6162 (2018)

  3. Chen, K., Pang, J., Wang, J., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Shi, J., Ouyang, W., et al.: Hybrid task cascade for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4974–4983 (2019)

  4. Chen, W., Sun, T., Li, M., Jiang, H., Zhou, C.: A new image co-segmentation method using saliency detection for surveillance image of coal miners. Comput. Electr. Eng. 40(8), 227–235 (2014)

    Article  Google Scholar 

  5. Chong, Y., Chen, W., Li, Z., Lam, W.H., Zheng, C., Li, Q.: Integrated real-time vision-based preceding vehicle detection in urban roads. Neurocomputing 116, 144–149 (2013)

    Article  Google Scholar 

  6. Dai, J., Li, Y., He, K., Sun, J.: R-fcn: Object detection via region-based fully convolutional networks. In: Advances in Neural Information Processing Systems, pp. 379–387 (2016)

  7. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: Deformable convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 764–773 (2017)

  8. Ding, J., Xue, N., Long, Y., Xia, G.S., Lu, Q.: Learning roi transformer for detecting oriented objects in aerial images (2018). arXiv preprint arXiv:1812.00155

  9. Ding, Q., Shang, J., Sun, Y., Wang, X., Liu, J.X.: Hc-hdsd: a method of hypergraph construction and high-density subgraph detection for inferring high-order epistatic interactions. Comput. Biol. Chem. 78, 440–447 (2019)

    Article  Google Scholar 

  10. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., Tian, Q.: Centernet: keypoint triplets for object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6569–6578 (2019)

  11. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  12. Girshick, R.: Fast r-cnn. In: The IEEE International Conference on Computer Vision (ICCV) (2015)

  13. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

  14. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

  15. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)

    Article  Google Scholar 

  16. Howse, J.: OpenCV Computer Vision with Python. Packt Publishing Ltd, Birmingham (2013)

    Google Scholar 

  17. Huang, L., Yang, Y., Deng, Y., Yu, Y.: Densebox: Unifying landmark localization with end to end object detection (2015). arXiv preprint arXiv:1509.04874

  18. Jiang, Y., Zhu, X., Wang, X., Yang, S., Li, W., Wang, H., Fu, P., Luo, Z.: R2cnn: rotational region cnn for orientation robust scene text detection (2017). arXiv preprint arXiv:1706.09579

  19. Kong, T., Sun, F., Liu, H., Jiang, Y., Shi, J.: Foveabox: beyond anchor-based object detector (2019). arXiv preprint arXiv:1904.03797

  20. Law, H., Deng, J.: Cornernet: detecting objects as paired keypoints. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 734–750 (2018)

  21. Li, S., Shang, J., Chen, Q., Sun, Y., Liu, J.X.: A compressed sensing based multi-stage method for detecting epistatic interactions. Int. J. Data Mining Bioinform. 14(4), 354–372 (2016)

    Article  Google Scholar 

  22. Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., Sun, J.: Light-head r-cnn: in defense of multi-stage object detector (2017). arXiv preprint arXiv:1711.07264

  23. Liang, Y., Cai, Z., Yu, J., Han, Q., Li, Y.: Deep learning based inference of private information using embedded sensors in smart devices. IEEE Netw. 32(4), 8–14 (2018)

    Article  Google Scholar 

  24. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

  25. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

  26. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European Conference on Computer Vision. Springer, pp. 740–755 (2014)

  27. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision. Springer, pp. 21–37 (2016)

  28. Ma, J., Shao, W., Ye, H., Wang, L., Wang, H., Zheng, Y., Xue, X.: Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans. Multimedia 20(11), 3111–3122 (2018)

    Article  Google Scholar 

  29. Ma, X., Zhang, F., Chen, X., Shen, J.: Privacy preserving multi-party computation delegation for deep learning in cloud computing. Inf. Sci. 459, 103–116 (2018)

    Article  Google Scholar 

  30. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017)

  31. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  32. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  33. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

  34. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks (2013). arXiv preprint arXiv:1312.6229

  35. Shang, J., Sun, Y., Liu, J.X., Xia, J., Zhang, J., Zheng, C.H.: Cinoedv: a co-information based method for detecting and visualizing n-order epistatic interactions. BMC Bioinform. 17(1), 214 (2016)

    Article  Google Scholar 

  36. Tian, Z., Shen, C., Chen, H., He, T.: Fcos: Fully convolutional one-stage object detection (2019). arXiv preprint arXiv:1904.01355

  37. Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: Dota: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3974–3983 (2018)

  38. Yang, X., Yang, J., Yan, J., Zhang, Y., Zhang, T., Guo, Z., Sun, X., Fu, K.: Scrdet: towards more robust detection for small, cluttered and rotated objects. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8232–8241 (2019)

  39. Yang, Z., Liu, S., Hu, H., Wang, L., Lin, S.: Reppoints: Point set representation for object detection (2019). arXiv preprint arXiv:1904.11490

  40. Zeng, Q., Martin, R.R., Wang, L., Quinn, J.A., Sun, Y., Tu, C.: Region-based bas-relief generation from a single image. Gr. Models 76(3), 140–151 (2014)

    Article  Google Scholar 

  41. Zhang, S., Wen, L., Bian, X., Lei, Z., Li, S.Z.: Single-shot refinement neural network for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4203–4212 (2018)

  42. Zhang, Z., Guo, W., Zhu, S., Yu, W.: Toward arbitrary-oriented ship detection with rotated region proposal and discrimination networks. IEEE Geosci. Remote Sens. Lett. 15(11), 1745–1749 (2018)

    Article  Google Scholar 

  43. Zheng, Y., Xu, X., Qi, L.: Deep cnn-assisted personalized recommendation over big data for mobile wireless networks. Wirel. Commun. Mob. Comput. 2019 (2019)

  44. Zhou, C., Liu, C.: Co-segmentation of multiple similar images using saliency detection and region merging. IET Comput. Vis. 8(3), 254–261 (2013)

    Article  Google Scholar 

  45. Zhou, C., Liu, C.: An efficient segmentation method using saliency object detection. Multimedia Tools Appl. 74(15), 5623–5634 (2015)

    Article  Google Scholar 

  46. Zhou, C., Wu, D., Qin, W., Liu, C.: An efficient multi-stage region merging method for interactive image segmentation. Comput. Electr. Eng. 54, 220–229 (2016)

    Article  Google Scholar 

  47. Zhou, X., Wang, D., Krähenbühl, P.: Objects as points (2019). arXiv preprint arXiv:1904.07850

  48. Zhou, X., Yao, C., Wen, H., Wang, Y., Zhou, S., He, W., Liang, J.: East: an efficient and accurate scene text detector. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 5551–5560 (2017)

  49. Zhu, C., He, Y., Savvides, M.: Feature selective anchor-free module for single-shot object detection (2019). arXiv preprint arXiv:1903.00621

  50. ZK, L., LB, W., YP, Y., et al.: A high resolution optical satellite image dataset for ship recognition and some new baselines (2017)

Download references

Acknowledgements

This work is supported in part by the National Natural Science Foundation of China under Grant nos. 61672254, 61802062, 61672246, 61572221, and 61300222, the Project of Department of Education of Guangdong Province of Grant number 2017KQNCX209, Key project of National Natural Science Foundation of China Grant no. U1536203, Natural Science Foundation of Hubei Province Grant no. 2015CFB687, the Fundamental Research Funds for the Central Universities, and HUST: 2016YXMS088 and 2016YXMS018. The authors appreciate the valuable suggestions from the anonymous reviewers and the Editors.

Author information

Authors and Affiliations

  1. Wuhan Digital Engineering Research Institute, Wuhan, China

    Zhixiang Yang & Wanhua Cao

  2. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China

    Fuhao Zou

  3. School of Natural and Computational Science, Massey University, Wellington, New Zealand

    Xiaoyun Jia

  4. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China

    Kunkun He & Kai Li

  5. School of Mathematics and Big Data, Foshan University, Foshan, China

    Chuntao Jiang

Authors
  1. Zhixiang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kunkun He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fuhao Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wanhua Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoyun Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chuntao Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuhao Zou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Z., He, K., Zou, F. et al. ROPDet: real-time anchor-free detector based on point set representation for rotating object. J Real-Time Image Proc 17, 2127–2138 (2020). https://doi.org/10.1007/s11554-020-01013-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-01013-7

Keywords

Search

Navigation