Abstract
Oil Spills near shoreline are a major environmental hazard. Rapid estimation of spill perimeter provides a quick estimate of its area extent thus facilitating its quick removal. In this study; a meta-heuristic algorithm; “Gamma-Levy Hybrid Meta-heuristic with Conditional Evolution (GLHM-CE)” is proposed. The proposed algorithm is then used to evolve a distributed control strategy for a swarm of unmanned aerial vehicles for rapid confinement and estimation of spill perimeter. Every agent is controlled by a Deep Q Network whose Hyper-Parameters are tuned by GLHM-CE. Evaluation of GLHM-CE over 28 Blackbox Problems of CEC-2013,Special Session on Real-Parameter Optimization and its comparison with evolutionary algorithms like SHADE,Co-DE and JADE reveals that GLHM-CE successfully evades local minima and has a fast convergence. The effectiveness in hyper-parameter tuning of a Deep Q Network by GLHM-CE was evaluated over the quintessential CartPole problem from OpenAI Gym framework.
Similar content being viewed by others
References
Al-Kaabi NS, Kristensen M, Zouari N, Solling TI, Bach SS, Al-Ghouti M, Christensen JH (2017) Source identification of beached oil at al zubarah, northwestern qatar. J Pet Sci Eng 149:107–113. https://doi.org/10.1016/j.petrol.2016.10.034
Banerjee A, Ghosh D, Das S (2018) Modified firefly algorithm for area estimation and tracking of fast expanding oil spills. Appl Soft Comput 73:829–847. https://doi.org/10.1016/j.asoc.2018.09.024
Bellman R (1957) Dynamic programming. Dover Publications, New York
Brockman G, Cheung V, Pettersson L, Schneider J, Schulman J, Tang J, Zaremba W (2016) Openai gym. http://arxiv.org/abs/1606.01540. Cite arXiv:1606.01540
Bruemmer DJ, Dudenhoeffer DD, McKay MD, Anderson MO (2002) A robotic swarm for spill finding and perimeter formation. Tech. rep, DTIC Document
cartpole v1. https://gym.openai.com/envs/CartPole-v1/
Chehresa S, Amirkhani A, Rezairad GA, Mosavi MR (2016) Optimum features selection for oil spill detection in sar image. J Indian Soc Remote Sens 44(5):775–787. https://doi.org/10.1007/s12524-016-0553-x
Clark J, Fierro R (2007) Mobile robotic sensors for perimeter detection and tracking. ISA Trans 46(1):3–13. https://doi.org/10.1016/j.isatra.2006.08.001
Cruz D, McClintock J, Perteet B, Orqueda OA, Cao Y, Fierro R (2007) Decentralized cooperative control-a multivehicle platform for research in networked embedded systems. IEEE Control Syst 27(3):58–78. https://doi.org/10.1109/MCS.2007.365004
Fingas M, Brown C (2014) Review of oil spill remote sensing. Marine Pollut Bull 83(1):9–23. https://doi.org/10.1016/j.marpolbul.2014.03.059
Fricke GK, Zhang G, Caccavale A, Li W, Garg DP (2010) An intelligent sensing network of distributed swarming agents for perimeter detection and surveillance. In: ASME 2010 Dynamic Systems and Control Conference, pp 741–748. American Society of Mechanical Engineers. https://doi.org/10.1115/DSCC2010-4256
Fujino S, Hatanaka T, Mori N, Matsumoto K (2019) Evolutionary deep learning based on deep convolutional neural network for anime storyboard recognition. Neurocomputing 338:393–398. https://doi.org/10.1016/j.neucom.2018.05.124
Gauci J, Stanley KO (2010) Autonomous evolution of topographic regularities in artificial neural networks. Neural Comput 22(7):1860–1898. https://doi.org/10.1162/neco.2010.06-09-1042 PMID: 20235822
Jha MN, Levy J, Gao Y (2008) Advances in remote sensing for oil spill disaster management: state-of-the-art sensors technology for oil spill surveillance. Sensors 8(1):236–255. https://doi.org/10.3390/s8010236
Karantzalos K, Argialas D (2008) Automatic detection and tracking of oil spills in sar imagery with level set segmentation. Int J Remote Sens 29(21):6281–6296. https://doi.org/10.1080/01431160802175488
LECUN Y The mnist database of handwritten digits. http://yann.lecun.com/exdb/mnist/https://ci.nii.ac.jp/naid/10027939599/en/
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature. https://doi.org/10.1038/nature14236
Morse G, Stanley KO. (2016) Simple evolutionary optimization can rival stochastic gradient descent in neural networks. In: Proceedings of the genetic and evolutionary computation conference 2016, GECCO ’16, p. 477-484. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2908812.2908916
Nirchio F, Sorgente M, Giancaspro A, Biamino W, Parisato E, Ravera R, Trivero P (2005) Automatic detection of oil spills from sar images. Int J Remote Sens 26(6):1157–1174. https://doi.org/10.1080/01431160512331326558
Ortego P, Diez-Olivan A, Ser JD, Veiga F, Penalva M, Sierra B (2020) Evolutionary LSTM-FCN networks for pattern classification in industrial processes. Swarm Evolu Comput 54:100650. https://doi.org/10.1016/j.swevo.2020.100650
Rumelhart DE, Hinton GE, Williams RJ (1986) Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1. chap. Learning Internal Representations by Error Propagation, pp 318–362. MIT Press, Cambridge, MA, USA. http://dl.acm.org/citation.cfm?id=104279.104293
Schaffer JD, Whitley D, Eshelman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks, pp 1–37
Shi X, Tian S, Yu L, Li L, Gao S (2017) Prediction of soil adsorption coefficient based on deep recursive neural network. Autom Control Comput Sci 51(5):321–330. https://doi.org/10.3103/S0146411617050066
Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127. https://doi.org/10.1162/106365602320169811
Suganuma M, Shirakawa S, Nagao T (2017) A genetic programming approach to designing convolutional neural network architectures. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’17, pp 497–504. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3071178.3071229
Tanabe R, Fukunaga A (2013) Success-history based parameter adaptation for differential evolution. In: 2013 IEEE Congress on Evolutionary Computation, pp 71–78. https://doi.org/10.1109/CEC.2013.6557555
Tran T, Yazdanparast A, Suess EA (2014) Effect of oil spill on birds: a graphical assay of the deepwater horizon oil spill’s impact on birds. Comput Statist 29(1):133–140. https://doi.org/10.1007/s00180-013-0472-z
Wang W, Yang J, Xiao J, Li S, Zhou D (2015) Face recognition based on deep learning. In: Zu Q, Hu B, Gu N, Seng S (eds) Human centered computing. Springer, Cham, pp 812–820
Wang Y, Cai Z, Zhang Q (2011) Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans Evolut Comput 15(1):55–66. https://doi.org/10.1109/TEVC.2010.2087271
Yang XS, Deb S (2013) Multiobjective cuckoo search for design optimization. Comput Oper Res 40(6):1616–1624. https://doi.org/10.1016/j.cor.2011.09.026
Yang Y, Li Y, Zhu X (2017) A novel oil spill detection method from synthetic aperture radar imageries via a bidimensional empirical mode decomposition. Acta Oceanologica Sinica 36(7):86–94. https://doi.org/10.1007/s13131-017-1086-z
Yu F, Hu X, Dong S, Liu G, Zhao Y, Chen G (2018) Design of a low-cost oil spill tracking buoy. J Marine Sci Technol 23(1):188–200. https://doi.org/10.1007/s00773-017-0472-8
Zambrano-Bigiarini M, Gonzalez-Fernandez Y (2015) cec2013: Benchmark Functions for the Special Session and Competition on Real-Parameter Single Objective Optimization at CEC-2013. http://CRAN.R-project.org/package=cec2013. R package version 0.1-5
Zelenke B, O’Connor C, Barker C, Beegle-Krause C, Eclipse L (2012) General noaa operational modeling environment (gnome) technical documentation: data formats. us dept. of commerce. NOAA Technical Memorandum NOS OR&R 41:49
Zhang J, Sanderson AC (2009) Jade: adaptive differential evolution with optional external archive. IEEE Trans Evolut Comput 13(5):945–958. https://doi.org/10.1109/TEVC.2009.2014613
Zhining Y, Yunming P (2015) The genetic convolutional neural network model based on random sample. Int J u-and e-Service Sci Technol 8(11):317–326
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Banerjee, A., Ghosh, D. & Das, S. Hyper-parameter tuned deep Q network for area estimation of oil spills: a meta-heuristic approach. Evol. Intel. 14, 175–190 (2021). https://doi.org/10.1007/s12065-020-00500-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12065-020-00500-x