Skip to main content
Springer Nature Link
Log in

Small-scale moving target detection in aerial image by deep inverse reinforcement learning

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

It proposes a deep inverse reinforcement learning method for slow and weak moving targets detection in aerial video. Differential gray images of adjacent frames are used as the network model input, and the feature network layer extracts the candidate moving target regions through the multi-layer convolution. The candidate target information is used as the initial layer of the policy network. The expert trajectory is used to adjust and optimize the feature convolution network model and the policy fully connected network model to realize the training the reward return function and the expert policy. In the stage of autonomous improvement policy, the policy model is re-optimized by unmarked aerial video, and deep inverse reinforcement learning and nonlinear policy network are used to make decision on moving target position and size information. The target size of the multi-group aerial video test set is 10 * 10 pixels. Experimental results show that the proposed algorithm has the advantage of the nonlinear policy of the neural network compared with the traditional moving target detection algorithm, and the detection result is more accurate. At the same time, compared with the traditional marginal programming (MMP) method and the structured classification based (SCIRL) method, the proposed algorithm shows obvious advantages in the accuracy of aerial video moving target detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Carneiro G, Nascimento JC (2013) Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data. IEEE Trans Pattern Anal Mach Intell 35(11):2592–2607

    Article  Google Scholar 

  • Chang X, Yang Y (2014) Semi-supervised feature analysis by mining correlations among multiple tasks. IEEE Trans Neural Netw Learn Syst 28(10):2294–2305

    Article  Google Scholar 

  • Chang X, Yu YL, Yang Y, Xing EP (2016) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Softw Eng 39(8):1617–1632

    Google Scholar 

  • Chang X, Ma Z, Lin M, Yang Y, Hauptmann A (2017a) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26(8):3911–3920

    Article  MathSciNet  MATH  Google Scholar 

  • Chang X, Ma Z, Yang Y, Zeng Z, Hauptmann AG (2017b) Bi-level semantic representation analysis for multimedia event detection. IEEE Trans Cybern 47(5):1180–1197

    Article  Google Scholar 

  • Chen C, Liu K, Kehtarnavaz N (2016) Real-time human action recognition based on depth motion maps. J Real-Time Image Proc 12(1):155–163

    Article  Google Scholar 

  • Choi J, Kim KE (2017) Hierarchical Bayesian inverse reinforcement learning. IEEE Trans Cybern 45(4):793–805

    Article  Google Scholar 

  • Dikmen O, Fevotte C (2012) Maximum marginal likelihood estimation for nonnegative dictionary learning in the gamma–Poisson model. IEEE Trans Signal Process 60(10):5163–5175

    Article  MathSciNet  MATH  Google Scholar 

  • Jeba JA, Roy S, Rashid MO et al (2019) Towards green cloud computing an algorithmic approach for energy minimization in cloud data centers. Int J Cloud Appl Comput 9(1):59–81

    Google Scholar 

  • Kelly JD, Hedengren JD (2013) A steady-state detection (SSD) algorithm to detect non-stationary drifts in processes. J Process Control 23(3):326–331

    Article  Google Scholar 

  • Khellah FM (2011) Texture classification using dominant neighborhood structure. IEEE Trans Image Process 20(11):3270–3279

    Article  MathSciNet  MATH  Google Scholar 

  • Konda V (2003) Actor-critic algorithms. SIAM J Control Optim 42(4):1143–1166

    Article  MathSciNet  MATH  Google Scholar 

  • Lazib L, Zhao Y, Qin B, Liu T (2016) Negation scope detection with recurrent neural networks models in review texts. In: International conference of young computer scientists, engineers and educators. Springer, Singapore

  • Li L, Zhu H, Yang G, Qian J (2014) Referenceless measure of blocking artifacts by Tchebichef kernel analysis. IEEE Signal Process Lett 21(1):122–125

    Article  Google Scholar 

  • Li L, Lin W, Wang X, Yang G, Bahrami K, Kot AC (2016a) No-reference image blur assessment based on discrete orthogonal moments. IEEE Trans Cybern 46(1):39–50

    Article  Google Scholar 

  • Li L, Wu D, Wu J, Li H, Lin W, Kot AC (2016b) Image sharpness assessment by sparse representation. IEEE Trans Multimed 18(6):1085–1097

    Article  Google Scholar 

  • Li Z, Nie F, Chang X, Yang Y (2017a) Beyond trace ratio: weighted harmonic mean of trace ratios for multiclass discriminant analysis. IEEE Transa Knowl Data Eng 29(10):2100–2110

    Article  Google Scholar 

  • Li L, Xia W, Lin W, Fang Y, Wang S (2017b) No-reference and robust image sharpness evaluation based on multiscale spatial and spectral features. IEEE Trans Multimed 19(5):1030–1040

    Article  Google Scholar 

  • Liao RF, Wen H, Wu J, Pan F, Xu A, Jiang Y, Cao M (2019) Deep-learning-based physical layer authentication for industrial wireless sensor networks. Sensors 19(11):2440

    Article  Google Scholar 

  • Lincoln R, Galloway S, Stephen B et al (2012) Comparing policy gradient and value function based reinforcement learning methods in simulated electrical power trade. IEEE Trans Power Syst 27(1):373–380

    Article  Google Scholar 

  • Mathews VJ, Xie Z (1993) A stochastic gradient adaptive filter with gradient adaptive step size. IEEE Trans Signal Process 41(6):2075–2087

    Article  MATH  Google Scholar 

  • Mnih V, Kavukcuoglu K, Silver D et al (2013) Playing Atari with deep reinforcement learning. Comput Sci 12:1–9

    Google Scholar 

  • Nair A, Srinivasan P, Blackwell S et al (2015) Massively parallel methods for deep reinforcement learning. Comput Sci

  • Nguyen P, Arsalan M, Koo J et al (2018) LightDenseYOLO: a fast and accurate marker tracker for autonomous UAV landing by visible light camera sensor on drone. Sensors 18(6):1315

    Article  Google Scholar 

  • Ozturk E, Sokmen I (2015) Resonant peaks of the linear optical absorption and rectification coefficients in GaAs/GaAlAs quantum well: combined effects of intense laser, electric and magnetic fields. Int J Mod Phys B 29(05):2338

    Article  Google Scholar 

  • Pan J-S, Kong L, Sung T-W, Tsai P-W, Snasel W (2018) α-fraction first strategy for hierarchical wireless sensor neteorks. J Internet Technol 19(6):1717–1726

    Google Scholar 

  • Sutton RS (1988) Learning to predict by the method of temporal differences. Mach Learn 3(1):9–44

    Google Scholar 

  • Van Hasselt H, Guez A, Silver D (2015) Deep reinforcement learning with double Q-learning. Comput Sci 9:1–9

    Article  Google Scholar 

  • Wu J, Guo S, Huang H, Liu W, Xiang Y (2018) Information and communications technologies for sustainable development goals: state-of-the-art, needs and perspectives. IEEE Commun Surv Tutor 20(3):2389–2406

    Article  Google Scholar 

  • Xia C, El Kamel A (2016) Neural inverse reinforcement learning in autonomous navigation. Robot Autonomous Syst 84:1–14

    Article  Google Scholar 

  • Yang Q, Xue D (2013) Gait recognition based on sparse representation and segmented frame difference energy image. Inf Control 42(1):27–32

    Article  Google Scholar 

  • Yang G et al (2018) Convolutional neural network-based embarrassing situation detection under camera for social robot in smart homes. Sensors 18(5):1530

    Article  Google Scholar 

  • Zeng X, Yeung DS (2001) Sensitivity analysis of multilayer perceptron to input and weight perturbations. IEEE Trans Neural Netw 12(6):1358–1366

    Article  Google Scholar 

  • Zhang Q, Liu Y, Pan J, Yan Y (2015) Continuous speech recognition based on convolutional neural network. In: International conference on digital image processing, international society for optics and photonics

  • Zhifei S, Joo EM (2012) A survey of inverse reinforcement learning techniques. Int J Intell Comput Cybern 5(3):293–311

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We would like to thank the anonymous reviewers and the associate editor for their valuable comments and suggestions to improve the quality of the manuscript. This work was supported by National Nature Science Foundation of China (NSFC) under Grants 61671356, 61703403, 61601352.

Author information

Authors and Affiliations

  1. School of Aerospace Science and Technology, Xidian University, Xi’an, 710071, China

    Wei Sun, Dashuai Yan & Jie Huang

  2. Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology, Beijing, 100094, China

    Changhao Sun

Authors
  1. Wei Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Dashuai Yan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jie Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Changhao Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Wei Sun.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Communicated by B. B. Gupta.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was funded by National Nature Science Foundation of China (NSFC) under Grants 61671356, 61703403.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, W., Yan, D., Huang, J. et al. Small-scale moving target detection in aerial image by deep inverse reinforcement learning. Soft Comput 24, 5897–5908 (2020). https://doi.org/10.1007/s00500-019-04404-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04404-6

Keywords