Skip to main content
SpringerLink
Log in

Infrared small target detection via region super resolution generative adversarial network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Infrared small target detection has always been a difficult problem in the field of object detection. The main reason affecting the accuracy is that the small infrared target has fewer pixels and weaker features. The current optimization methods for the small target are mainly based on multi-scale feature fusion or super-resolution enhancement. When super-resolution networks are applied to infrared target detection, there are still two non-negligible problems: first, the super-resolution structure will consume too much arithmetic power, resulting in a low detection rate, and second, the low-resolution images characterizing small targets are usually obtained by downsampling with high-resolution images during training, which is different from the distribution of tiny target in actual detection applications, resulting in poor detection accuracy. We propose a new detection network to solve the above problem: Region Super Resolution Generative Adversarial Network(RSRGAN). It contains a simple structured network, Region Context Network(RCN) as the backbone that consumes less computational cost to extract the possible regions. The generator of the Generative Adversarial Network(GAN) includes two modules: distribution transformation and super-resolution enhancement. First, the blurred infrared small target is converted into a clear target with similar distribution as the training set. Then the resolution is increased, which can achieve a better enhancement effect. The discriminator distinguishes whether the input comes from the generator or the actual image to assist in generating a better super-resolution image. Meanwhile, we produced an infrared Unmanned Aerial Vehicle(UAV) small target dataset, target pixels below 20*20, containing birds, leaves, and other similar disturbances, which is more challenging for the detection algorithm. Our method proves better detection of small IR targets and shows superior performance over state-of-the-art methods through experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Zhang D, Zhang J, Yao K et al (2016) Infrared ship-target recognition based on SVM classification[J]. Infrared and Laser Engineering 45(1):1–6

    Article  MathSciNet  Google Scholar 

  2. Ding S, Guo L, Hou Y (2017) Extreme learning machine with kernel model based on deep learning[J]. Neural Comput & Applic 28(8):1975–1984

    Article  Google Scholar 

  3. Yang C, Yao J, Sun D et al (2016) Kernel sparse coding method for automatic target recognition in infrared imagery using covariance descriptor[J]. Infrared Phys Technol 76:740–747

    Article  Google Scholar 

  4. Redmon J, Farhadi A. Yolov3: An incremental improvement[J]

  5. Bochkovskiy A, Wang C Y, Liao H Y M. Yolov4: Optimal speed and accuracy of object detection[J]

  6. Liu W, Anguelov D, Erhan D et al (2016) Ssd: single shot multibox detector[C]//European conference on computer vision. Springer, Cham, pp 21–37

    Google Scholar 

  7. Ren S, He K, Girshick R et al Faster r-cnn: Towards real-time object detection with region proposal networks[J]

  8. Fu C Y, Liu W, Ranga A et al (2017) Dssd: Deconvolutional single shot detector[J]. arXiv preprint arXiv:1701.06659

  9. Lin T Y, Dollár P, Girshick R et al (2017) Feature pyramid networks for object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  10. Li J, Liang X, Wei Y, et al (2017) Perceptual generative adversarial networks for small object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1222–1230

  11. Bai Y, Zhang Y, Ding M et al (2018) Finding tiny faces in the wild with generative adversarial network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 21–30

  12. Bai Y, Zhang Y, Ding M et al (2018) Sod-mtgan: Small object detection via multi-task generative adversarial network[C]//Proceedings of the European Conference on Computer Vision (ECCV). pp 206–221

  13. Wang J, Zhang W, Cao Y et al (2020) Side-aware boundary localization for more precise object detection[C]//European conference on computer vision. Springer, Cham, pp 403–419

    Google Scholar 

  14. Wang S, Gong Y, Xing J et al (2020) Rdsnet: A new deep architecture forreciprocal object detection and instance segmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 34(07): 12208–12215

  15. Girshick R, Donahue J, Darrell T et al (2014) Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  16. Girshick R (2015) Fast r-cnn[C]//Proceedings of the IEEE international conference on computer vision, 1440–1448

  17. He K, Zhang X, Ren S et al (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  18. Dai J, Li Y, He K et al (2016) R-fcn: Object detection via region-based fully convolutional networks[J]. arXiv preprint arXiv:1605.06409

  19. Ghiasi G, Lin T Y, Le Q V (2019) Nas-fpn: Learning scalable feature pyramid architecture for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7036–7045

  20. Xu H, Yao L, Zhang W et al (2019) Auto-fpn: Automatic network architecture adaptation for object detection beyond classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 6649–6658

  21. Redmon J, Divvala S, Girshick R et al (2016) You only look once: Unified, real-time object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  22. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271

  23. Lin T Y, Goyal P, Girshick R et al (2017) Focal loss for dense object detection[C]//Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  24. Xu Y, Sun B, Yan X et al (2020) Multi-focus image fusion using learning based matting with sum of the Gaussian-based modified Laplacian[J]. Digital Signal Processing 106:102821

    Article  Google Scholar 

  25. Xu Y, Sun B (2020) Color-compensated multi-scale exposure fusion based on physical features[J]. Optik 223:165494

    Article  Google Scholar 

  26. Yan X, Liu Y, Xu Y et al (2020) Multistep forecasting for diurnal wind speed based on hybrid deep learning model with improved singular spectrum decomposition[J]. Energy Convers Manag 225:113456

    Article  Google Scholar 

  27. Xu Y, Yang C, Sun B et al (2021) A novel multi-scale fusion framework for detail-preserving low-light image enhancement[J]. Inf Sci 548:378–397

    Article  MathSciNet  Google Scholar 

  28. Tan M, Pang R, Le Q V (2020) Efficientdet: Scalable and efficient object detection[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10781–10790

  29. Ledig C, Theis L, Huszár F et al (2017) Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  30. Goodfellow I J, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial networks[J]. arXiv preprint arXiv:1406.2661

  31. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint arXiv:1511.06434

  32. Zhu J Y, Park T, Isola P et al (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE international conference on computer vision, pp 2223–2232

  33. Brock A, Donahue J, Simonyan K (2018) Large scale GAN training for high fidelity natural image synthesis[J]. arXiv preprint arXiv:1809.11096

  34. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 62175111 and 62001234, in part by the Natural Science Foundation of Jiangsu Province under Grant BK20200487, in part by the China Postdoctoral Science Foundation under Grant 2020 M681597, in part by the Postdoctoral Science Foundation of Jiangsu Province under Grant 2020Z051, and in part by the Shanghai Aerospace Science and Technology Innovation Foundation under Grant SAST2020-071.

Author information

Authors and Affiliations

  1. Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense, Nanjing University of Science and Technology, Nanjing, 210094, China

    Kan Ren, Yuan Gao, Minjie Wan, Guohua Gu & Qian Chen

Authors
  1. Kan Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minjie Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guohua Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qian Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kan Ren.

Ethics declarations

Conflict of interest

The author declares that they have no conflict of interests concerning the content of this study.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, K., Gao, Y., Wan, M. et al. Infrared small target detection via region super resolution generative adversarial network. Appl Intell 52, 11725–11737 (2022). https://doi.org/10.1007/s10489-021-02955-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02955-6

Keywords

Search

Navigation