Skip to main content
SpringerLink
Log in

Hyper-parameter tuned deep Q network for area estimation of oil spills: a meta-heuristic approach

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

Oil Spills near shoreline are a major environmental hazard. Rapid estimation of spill perimeter provides a quick estimate of its area extent thus facilitating its quick removal. In this study; a meta-heuristic algorithm; “Gamma-Levy Hybrid Meta-heuristic with Conditional Evolution (GLHM-CE)” is proposed. The proposed algorithm is then used to evolve a distributed control strategy for a swarm of unmanned aerial vehicles for rapid confinement and estimation of spill perimeter. Every agent is controlled by a Deep Q Network whose Hyper-Parameters are tuned by GLHM-CE. Evaluation of GLHM-CE over 28 Blackbox Problems of CEC-2013,Special Session on Real-Parameter Optimization and its comparison with evolutionary algorithms like SHADE,Co-DE and JADE reveals that GLHM-CE successfully evades local minima and has a fast convergence. The effectiveness in hyper-parameter tuning of a Deep Q Network by GLHM-CE was evaluated over the quintessential CartPole problem from OpenAI Gym framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Al-Kaabi NS, Kristensen M, Zouari N, Solling TI, Bach SS, Al-Ghouti M, Christensen JH (2017) Source identification of beached oil at al zubarah, northwestern qatar. J Pet Sci Eng 149:107–113. https://doi.org/10.1016/j.petrol.2016.10.034

    Article  Google Scholar 

  2. Banerjee A, Ghosh D, Das S (2018) Modified firefly algorithm for area estimation and tracking of fast expanding oil spills. Appl Soft Comput 73:829–847. https://doi.org/10.1016/j.asoc.2018.09.024

    Article  Google Scholar 

  3. Bellman R (1957) Dynamic programming. Dover Publications, New York

    MATH  Google Scholar 

  4. Brockman G, Cheung V, Pettersson L, Schneider J, Schulman J, Tang J, Zaremba W (2016) Openai gym. http://arxiv.org/abs/1606.01540. Cite arXiv:1606.01540

  5. Bruemmer DJ, Dudenhoeffer DD, McKay MD, Anderson MO (2002) A robotic swarm for spill finding and perimeter formation. Tech. rep, DTIC Document

  6. cartpole v1. https://gym.openai.com/envs/CartPole-v1/

  7. Chehresa S, Amirkhani A, Rezairad GA, Mosavi MR (2016) Optimum features selection for oil spill detection in sar image. J Indian Soc Remote Sens 44(5):775–787. https://doi.org/10.1007/s12524-016-0553-x

    Article  Google Scholar 

  8. Clark J, Fierro R (2007) Mobile robotic sensors for perimeter detection and tracking. ISA Trans 46(1):3–13. https://doi.org/10.1016/j.isatra.2006.08.001

    Article  Google Scholar 

  9. Cruz D, McClintock J, Perteet B, Orqueda OA, Cao Y, Fierro R (2007) Decentralized cooperative control-a multivehicle platform for research in networked embedded systems. IEEE Control Syst 27(3):58–78. https://doi.org/10.1109/MCS.2007.365004

    Article  Google Scholar 

  10. Fingas M, Brown C (2014) Review of oil spill remote sensing. Marine Pollut Bull 83(1):9–23. https://doi.org/10.1016/j.marpolbul.2014.03.059

    Article  Google Scholar 

  11. Fricke GK, Zhang G, Caccavale A, Li W, Garg DP (2010) An intelligent sensing network of distributed swarming agents for perimeter detection and surveillance. In: ASME 2010 Dynamic Systems and Control Conference, pp 741–748. American Society of Mechanical Engineers. https://doi.org/10.1115/DSCC2010-4256

  12. Fujino S, Hatanaka T, Mori N, Matsumoto K (2019) Evolutionary deep learning based on deep convolutional neural network for anime storyboard recognition. Neurocomputing 338:393–398. https://doi.org/10.1016/j.neucom.2018.05.124

    Article  Google Scholar 

  13. Gauci J, Stanley KO (2010) Autonomous evolution of topographic regularities in artificial neural networks. Neural Comput 22(7):1860–1898. https://doi.org/10.1162/neco.2010.06-09-1042 PMID: 20235822

    Article  MATH  Google Scholar 

  14. Jha MN, Levy J, Gao Y (2008) Advances in remote sensing for oil spill disaster management: state-of-the-art sensors technology for oil spill surveillance. Sensors 8(1):236–255. https://doi.org/10.3390/s8010236

    Article  Google Scholar 

  15. Karantzalos K, Argialas D (2008) Automatic detection and tracking of oil spills in sar imagery with level set segmentation. Int J Remote Sens 29(21):6281–6296. https://doi.org/10.1080/01431160802175488

    Article  Google Scholar 

  16. LECUN Y The mnist database of handwritten digits. http://yann.lecun.com/exdb/mnist/https://ci.nii.ac.jp/naid/10027939599/en/

  17. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature. https://doi.org/10.1038/nature14236

    Article  Google Scholar 

  18. Morse G, Stanley KO. (2016) Simple evolutionary optimization can rival stochastic gradient descent in neural networks. In: Proceedings of the genetic and evolutionary computation conference 2016, GECCO ’16, p. 477-484. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2908812.2908916

  19. Nirchio F, Sorgente M, Giancaspro A, Biamino W, Parisato E, Ravera R, Trivero P (2005) Automatic detection of oil spills from sar images. Int J Remote Sens 26(6):1157–1174. https://doi.org/10.1080/01431160512331326558

    Article  Google Scholar 

  20. Ortego P, Diez-Olivan A, Ser JD, Veiga F, Penalva M, Sierra B (2020) Evolutionary LSTM-FCN networks for pattern classification in industrial processes. Swarm Evolu Comput 54:100650. https://doi.org/10.1016/j.swevo.2020.100650

    Article  Google Scholar 

  21. Rumelhart DE, Hinton GE, Williams RJ (1986) Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1. chap. Learning Internal Representations by Error Propagation, pp 318–362. MIT Press, Cambridge, MA, USA. http://dl.acm.org/citation.cfm?id=104279.104293

  22. Schaffer JD, Whitley D, Eshelman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks, pp 1–37

  23. Shi X, Tian S, Yu L, Li L, Gao S (2017) Prediction of soil adsorption coefficient based on deep recursive neural network. Autom Control Comput Sci 51(5):321–330. https://doi.org/10.3103/S0146411617050066

    Article  Google Scholar 

  24. Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127. https://doi.org/10.1162/106365602320169811

    Article  Google Scholar 

  25. Suganuma M, Shirakawa S, Nagao T (2017) A genetic programming approach to designing convolutional neural network architectures. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’17, pp 497–504. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3071178.3071229

  26. Tanabe R, Fukunaga A (2013) Success-history based parameter adaptation for differential evolution. In: 2013 IEEE Congress on Evolutionary Computation, pp 71–78. https://doi.org/10.1109/CEC.2013.6557555

  27. Tran T, Yazdanparast A, Suess EA (2014) Effect of oil spill on birds: a graphical assay of the deepwater horizon oil spill’s impact on birds. Comput Statist 29(1):133–140. https://doi.org/10.1007/s00180-013-0472-z

    Article  MathSciNet  MATH  Google Scholar 

  28. Wang W, Yang J, Xiao J, Li S, Zhou D (2015) Face recognition based on deep learning. In: Zu Q, Hu B, Gu N, Seng S (eds) Human centered computing. Springer, Cham, pp 812–820

    Chapter  Google Scholar 

  29. Wang Y, Cai Z, Zhang Q (2011) Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans Evolut Comput 15(1):55–66. https://doi.org/10.1109/TEVC.2010.2087271

    Article  Google Scholar 

  30. Yang XS, Deb S (2013) Multiobjective cuckoo search for design optimization. Comput Oper Res 40(6):1616–1624. https://doi.org/10.1016/j.cor.2011.09.026

    Article  MathSciNet  MATH  Google Scholar 

  31. Yang Y, Li Y, Zhu X (2017) A novel oil spill detection method from synthetic aperture radar imageries via a bidimensional empirical mode decomposition. Acta Oceanologica Sinica 36(7):86–94. https://doi.org/10.1007/s13131-017-1086-z

    Article  Google Scholar 

  32. Yu F, Hu X, Dong S, Liu G, Zhao Y, Chen G (2018) Design of a low-cost oil spill tracking buoy. J Marine Sci Technol 23(1):188–200. https://doi.org/10.1007/s00773-017-0472-8

    Article  Google Scholar 

  33. Zambrano-Bigiarini M, Gonzalez-Fernandez Y (2015) cec2013: Benchmark Functions for the Special Session and Competition on Real-Parameter Single Objective Optimization at CEC-2013. http://CRAN.R-project.org/package=cec2013. R package version 0.1-5

  34. Zelenke B, O’Connor C, Barker C, Beegle-Krause C, Eclipse L (2012) General noaa operational modeling environment (gnome) technical documentation: data formats. us dept. of commerce. NOAA Technical Memorandum NOS OR&R 41:49

    Google Scholar 

  35. Zhang J, Sanderson AC (2009) Jade: adaptive differential evolution with optional external archive. IEEE Trans Evolut Comput 13(5):945–958. https://doi.org/10.1109/TEVC.2009.2014613

    Article  Google Scholar 

  36. Zhining Y, Yunming P (2015) The genetic convolutional neural network model based on random sample. Int J u-and e-Service Sci Technol 8(11):317–326

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication, Dr. B C Roy Enginering College, Durgapur, India

    Abhiit Banerjee

  2. Department of Computer Science, Dr. B C Roy Enginering College, Durgapur, India

    Dipendranath Ghosh

  3. Department of Computer Science and Engineering, National Institute of Technology, Durgapur, India

    Suvrojit Das

Authors
  1. Abhiit Banerjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dipendranath Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Suvrojit Das
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abhiit Banerjee.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Banerjee, A., Ghosh, D. & Das, S. Hyper-parameter tuned deep Q network for area estimation of oil spills: a meta-heuristic approach. Evol. Intel. 14, 175–190 (2021). https://doi.org/10.1007/s12065-020-00500-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-020-00500-x

Keywords

Search

Navigation