Abstract
In the recent era of evolution in deep learning, several ANN techniques have been developed to forecast weather conditions. Temperature forecasting is a subset of weather forecasting which employ the use of these ANN techniques to predict the future trend in temperature variation. The modern ANN technologies have facilitated the prediction process due to its virtue of handling lengthy sequences which have been encorporated by many reserachers to train their models. These ANN models require an optimisation technique to mitigate the error between the forecasted and the true value, thus a lot of exploration has been done in the improvement of these optimisation strategies. In this research work, a unique derivative free optimiser known as Unscented Kalman Filter (UKF) is implemented for optimisation along with an upgraded form of Recurrent Neural Network called Long Short Term Memory (LSTM) to forecast the Global Land-Ocean index temperature. In the proposed model, LSTM is used as a base neural network due to its probity to handle very long sequences of data and resolving the issue of vanishing and exploding gradients suffered by RNN. The prediction results obtained from the propsed LSTM-UKF model was compared with Gradient Descent (GD) optimizers such as, Adam optimizer, RMSprop optimizer, and an evolutionary optimisation algorithm called Genetic Algorithm (GA) to demostrate its optimising strength. Well established predictive models such as Gated Recurrent Unit (GRU) and Convolutional Neural Network (CNN) were also trained using UKF and the results were compared with the base model along with the stated optimisation techniques.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) Tensorflow: a system for large-scale machine learning. In: 12th \(\{\)USENIX\(\}\) symposium on operating systems design and implementation (\(\{\)OSDI\(\}\) 16), pp 265–283
Abualigah LMQ (2019) Feature selection and enhanced krill herd algorithm for text document clustering. Springer, Berlin/Heidelberg
Abualigah LM, Khader AT, Hanandeh ES (2018) A new feature selection method to improve the document clustering using particle swarm optimization algorithm. J Comput Sci 25:456–466. ISSN 1877-7503. https://doi.org/10.1016/j.jocs.2017.07.018
Adem K, Kiliçarslan S (2019) Performance analysis of optimization algorithms on stacked autoencoder. In: 2019 3rd international symposium on multidisciplinary studies and innovative technologies (ISMSIT), pp 1–4. https://doi.org/10.1109/ISMSIT.2019.8932880
Almansouri H, Venkatakrishnan SV, Buzzard GT, Bouman CA, Santos-Villalobos H (2018) Deep neural networks for non-linear model-based ultrasound reconstruction. In: 2018 IEEE global conference on signal and information processing (GlobalSIP). IEEE, pp 6–10
Anderson JL (2009) Ensemble kalman filters for large geophysical applications. IEEE Control Syst Magaz 29(3):66–82
Banzhaf W, Koza JR, Ryan C, Spector L, Jacob C (2000) Genetic programming. IEEE Intell Syst Appl 15(3):74–84. https://doi.org/10.1109/5254.846288
Chen J, Chen W, Huang C, Huang S, Chen A (2016) Financial time-series data analysis using deep convolutional neural networks. In: 2016 7th international conference on cloud computing and big data (CCBD), pp 87–92. https://doi.org/10.1109/CCBD.2016.027
Chen C, Lin X, Huang Y, Terejanu G (2019) Approximate bayesian neural network trained with ensemble kalman filter. In: 2019 international joint conference on neural networks (IJCNN). IEEE, pp 1–8
Chollet F (2015) keras. https://github.com/fchollet/keras
Chung J, Gulcehre C, Cho KH, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. Preprint at arXiv:1412.3555
Deng W, Xu J, Zhao H (2019) An improved ant colony optimization algorithm based on hybrid strategies for scheduling problem. IEEE Access 7:20281–20292. https://doi.org/10.1109/ACCESS.2019.2897580
Gad A (2020) 5 genetic algorithm applications using pygad, 06
Hansen J, Lebedeff S (1987) Global trends of measured surface air temperature. J Geophys Res: Atmos 92(D11):13345–13372
Hansun S, Kristanda MB (2017) Performance analysis of conventional moving average methods in forex forecasting. In: 2017 international conference on smart cities, automation & intelligent computing systems (ICON-SONICS). IEEE, pp 11–17
Hochreiter S, Schmidhuber J (1997) Lstm can solve hard long time lag problems. In: Advances in neural information processing systems, pp 473–479
Immanuel SD, Chakraborty UK (2019) Genetic algorithm: an approach on optimization. In: 2019 international conference on communication and electronics systems (ICCES), pp 701–708. https://doi.org/10.1109/ICCES45898.2019.9002372
Kaur S, Rakshit M (2019) Predictability and analysis of weekly temperature using autoregressive time series models. In: 2019 international conference on computing, power and communication technologies (GUCON). IEEE, pp 616–621
Kawaguchi K (2016) Deep learning without poor local minima
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Preprint at arXiv:1412.6980
Lakshmivarahan S, Stensrud DJ (2009) Ensemble kalman filter. IEEE Control Syst Magaz 29(3):34–46
Li LK (1992) Approximation theory and recurrent networks. In: [Proceedings 1992] IJCNN international joint conference on neural networks, vol 2, pp 266–271. https://doi.org/10.1109/IJCNN.1992.226997
Li X, Wang K, Wang W, Li Y (2010) A multiple object tracking method using kalman filter. In: The 2010 IEEE international conference on information and automation. IEEE, pp 1862–1866
Mishra M, Srivastava M (2014) A view of artificial neural network. In: 2014 international conference on advances in engineering & technology research (ICAETR-2014). IEEE, pp 1–3
Ngo TB, Le HL, Nguyen TH (2009) Survey of kalman filters and their application in signal processing. In: 2009 international conference on artificial intelligence and computational intelligence, vol 3. IEEE, pp 335–339
Pascanu R, Mikolov T, Bengio Y (2013) On the difficulty of training recurrent neural networks. In: International conference on machine learning, pp 1310–1318
Puskorius GV, Feldkamp LA (1994) Neurocontrol of nonlinear dynamical systems with kalman filter trained recurrent networks. IEEE Trans Neural Netw 5(2):279–297
Ruder S (2016) An overview of gradient descent optimization algorithms. Preprint at arXiv:1609.04747
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536
Simsek B, Erdogan AT (2019) Online bounded component analysis: a simple recurrent neural network with local update rule for unsupervised separation of dependent and independent sources. In: 2019 53rd asilomar conference on signals, systems, and computers. IEEE, pp 1639–1643
Śmieszek M, Dobrzańska M (2015) Application of kalman filter in navigation process of automated guided vehicles. Metrol Measur Syst 22(3)
Tanan SC, Nathwani K, Jain A, Hegde RM, Rani R, Tripathy A (2014) Acoustic echo and noise cancellation using kalman filter in a modified gsc framework. In: 2014 48th asilomar conference on signals, systems and computers. IEEE, pp 477–481
Wan EA, Van Der Merwe R (2000) The unscented kalman filter for nonlinear estimation. In: Proceedings of the IEEE 2000 adaptive systems for signal processing, communications, and control symposium (Cat. No. 00EX373). IEEE, pp 153–158
Werbos PJ (1990) Backpropagation through time: what it does and how to do it. Proc IEEE 78(10):1550–1560
Yao S, Luo L, Peng H (2018) High-frequency stock trend forecast using lstm model. In: 2018 13th international conference on computer science education (ICCSE), pp 1–4. https://doi.org/10.1109/ICCSE.2018.8468703
Zhan R, Wan J (2006) Neural network-aided adaptive unscented kalman filter for nonlinear state estimation. IEEE Signal Process Lett 13(7):445–448
Zhang Q, Wang H, Dong J, Zhong G, Sun X (2017) Prediction of sea surface temperature using long short-term memory. IEEE Geosci Remote Sens Lett 14(10):1745–1749
Zhang H, Zhang L, Jiang Y (2019) Overfitting and underfitting analysis for deep learning based end-to-end communication systems. In: 2019 11th international conference on wireless communications and signal processing (WCSP), pp 1–6. https://doi.org/10.1109/WCSP.2019.8927876
Zhang K, Geng X, Yan X-H (2020) Prediction of 3-d ocean temperature by multilayer convolutional lstm. IEEE Geosci Remote Sens Lett
Zhao Y, Shen L (2011) Application of time series auto regressive model in price forecast. In: 2011 international conference on business management and electronic information, vol 4. IEEE, pp 768–771
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bera, S., Gupta, G.S., Kumar, D. et al. LSTM-UKF framework for an effective global land-ocean index temperature prediction. J Ambient Intell Human Comput 14, 2369–2384 (2023). https://doi.org/10.1007/s12652-022-04491-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-022-04491-8