Abstract
Infrared small target detection has always been a difficult problem in the field of object detection. The main reason affecting the accuracy is that the small infrared target has fewer pixels and weaker features. The current optimization methods for the small target are mainly based on multi-scale feature fusion or super-resolution enhancement. When super-resolution networks are applied to infrared target detection, there are still two non-negligible problems: first, the super-resolution structure will consume too much arithmetic power, resulting in a low detection rate, and second, the low-resolution images characterizing small targets are usually obtained by downsampling with high-resolution images during training, which is different from the distribution of tiny target in actual detection applications, resulting in poor detection accuracy. We propose a new detection network to solve the above problem: Region Super Resolution Generative Adversarial Network(RSRGAN). It contains a simple structured network, Region Context Network(RCN) as the backbone that consumes less computational cost to extract the possible regions. The generator of the Generative Adversarial Network(GAN) includes two modules: distribution transformation and super-resolution enhancement. First, the blurred infrared small target is converted into a clear target with similar distribution as the training set. Then the resolution is increased, which can achieve a better enhancement effect. The discriminator distinguishes whether the input comes from the generator or the actual image to assist in generating a better super-resolution image. Meanwhile, we produced an infrared Unmanned Aerial Vehicle(UAV) small target dataset, target pixels below 20*20, containing birds, leaves, and other similar disturbances, which is more challenging for the detection algorithm. Our method proves better detection of small IR targets and shows superior performance over state-of-the-art methods through experiments.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Zhang D, Zhang J, Yao K et al (2016) Infrared ship-target recognition based on SVM classification[J]. Infrared and Laser Engineering 45(1):1–6
Ding S, Guo L, Hou Y (2017) Extreme learning machine with kernel model based on deep learning[J]. Neural Comput & Applic 28(8):1975–1984
Yang C, Yao J, Sun D et al (2016) Kernel sparse coding method for automatic target recognition in infrared imagery using covariance descriptor[J]. Infrared Phys Technol 76:740–747
Redmon J, Farhadi A. Yolov3: An incremental improvement[J]
Bochkovskiy A, Wang C Y, Liao H Y M. Yolov4: Optimal speed and accuracy of object detection[J]
Liu W, Anguelov D, Erhan D et al (2016) Ssd: single shot multibox detector[C]//European conference on computer vision. Springer, Cham, pp 21–37
Ren S, He K, Girshick R et al Faster r-cnn: Towards real-time object detection with region proposal networks[J]
Fu C Y, Liu W, Ranga A et al (2017) Dssd: Deconvolutional single shot detector[J]. arXiv preprint arXiv:1701.06659
Lin T Y, Dollár P, Girshick R et al (2017) Feature pyramid networks for object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125
Li J, Liang X, Wei Y, et al (2017) Perceptual generative adversarial networks for small object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1222–1230
Bai Y, Zhang Y, Ding M et al (2018) Finding tiny faces in the wild with generative adversarial network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 21–30
Bai Y, Zhang Y, Ding M et al (2018) Sod-mtgan: Small object detection via multi-task generative adversarial network[C]//Proceedings of the European Conference on Computer Vision (ECCV). pp 206–221
Wang J, Zhang W, Cao Y et al (2020) Side-aware boundary localization for more precise object detection[C]//European conference on computer vision. Springer, Cham, pp 403–419
Wang S, Gong Y, Xing J et al (2020) Rdsnet: A new deep architecture forreciprocal object detection and instance segmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 34(07): 12208–12215
Girshick R, Donahue J, Darrell T et al (2014) Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587
Girshick R (2015) Fast r-cnn[C]//Proceedings of the IEEE international conference on computer vision, 1440–1448
He K, Zhang X, Ren S et al (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916
Dai J, Li Y, He K et al (2016) R-fcn: Object detection via region-based fully convolutional networks[J]. arXiv preprint arXiv:1605.06409
Ghiasi G, Lin T Y, Le Q V (2019) Nas-fpn: Learning scalable feature pyramid architecture for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7036–7045
Xu H, Yao L, Zhang W et al (2019) Auto-fpn: Automatic network architecture adaptation for object detection beyond classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 6649–6658
Redmon J, Divvala S, Girshick R et al (2016) You only look once: Unified, real-time object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788
Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271
Lin T Y, Goyal P, Girshick R et al (2017) Focal loss for dense object detection[C]//Proceedings of the IEEE international conference on computer vision, pp 2980–2988
Xu Y, Sun B, Yan X et al (2020) Multi-focus image fusion using learning based matting with sum of the Gaussian-based modified Laplacian[J]. Digital Signal Processing 106:102821
Xu Y, Sun B (2020) Color-compensated multi-scale exposure fusion based on physical features[J]. Optik 223:165494
Yan X, Liu Y, Xu Y et al (2020) Multistep forecasting for diurnal wind speed based on hybrid deep learning model with improved singular spectrum decomposition[J]. Energy Convers Manag 225:113456
Xu Y, Yang C, Sun B et al (2021) A novel multi-scale fusion framework for detail-preserving low-light image enhancement[J]. Inf Sci 548:378–397
Tan M, Pang R, Le Q V (2020) Efficientdet: Scalable and efficient object detection[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10781–10790
Ledig C, Theis L, Huszár F et al (2017) Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690
Goodfellow I J, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial networks[J]. arXiv preprint arXiv:1406.2661
Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint arXiv:1511.06434
Zhu J Y, Park T, Isola P et al (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE international conference on computer vision, pp 2223–2232
Brock A, Donahue J, Simonyan K (2018) Large scale GAN training for high fidelity natural image synthesis[J]. arXiv preprint arXiv:1809.11096
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 62175111 and 62001234, in part by the Natural Science Foundation of Jiangsu Province under Grant BK20200487, in part by the China Postdoctoral Science Foundation under Grant 2020 M681597, in part by the Postdoctoral Science Foundation of Jiangsu Province under Grant 2020Z051, and in part by the Shanghai Aerospace Science and Technology Innovation Foundation under Grant SAST2020-071.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author declares that they have no conflict of interests concerning the content of this study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ren, K., Gao, Y., Wan, M. et al. Infrared small target detection via region super resolution generative adversarial network. Appl Intell 52, 11725–11737 (2022). https://doi.org/10.1007/s10489-021-02955-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02955-6