skip to main content
10.1145/3440943.3444723acmconferencesArticle/Chapter ViewAbstractPublication PagesiceaConference Proceedingsconference-collections
research-article

An Effective Optimizer based on Global and Local Searched Experiences for Short-term Electricity Consumption Forecasting

Published: 27 September 2021 Publication History

Abstract

A precise forecasting for the future short-term electricity consumption will be quite useful for making a good plan for the power demand management. Since deep neural network (DNN) provides an effective way for the short-term load forecasting, one of the focuses of this research is thus to use it to construct the prediction model. The gradient-based optimizers (GBOs) have been widely used in DNN algorithms in recent years; however, they make it easy for DNN to trap in poor local regions during the training process. In this research, we propose a new optimizer that is based on the searched experiences to solve this problem to enhance the performance of GBOs. More precisely, the proposed optimizer integrates the best position searched, Lévy flight, and gradient descent to preserve not only the diversification but also the intensification of search during the training process. To evaluate the performance of the proposed optimizer, we compare it with several state-of-the-art GBOs for DNN; namely, Adagrad, RMSprop, and Adam, for training a forecasting model for the short-term electricity consumption forecasting problem. The simulation results show that the proposed optimizer outperforms all the other GBOs in terms of the mean absolute percentage error.

References

[1]
Ashfaq Ahmad, Nadeem Javaid, Mohsen Guizani, Nabil Alrajeh, and Zahoor Ali Khan. 2016. An accurate and fast converging short-term load forecasting model for industrial applications in a smart grid. IEEE Transactions on Industrial Informatics 13, 5 (2016), 2587--2596.
[2]
Ibrahim Aljarah, Hossam Faris, and Seyedali Mirjalili. 2018. Optimizing connection weights in neural networks using the whale optimization algorithm. Soft Computing 22, 1 (2018), 1--15.
[3]
Shima Amirsadri, Seyed Jalaleddin Mousavirad, and Hossein Ebrahimpour-Komleh. 2018. A Levy flight-based grey wolf optimizer combined with back-propagation algorithm for neural network training. Neural Computing and Applications 30, 12 (2018), 3707--3720.
[4]
Alireza Askarzadeh and Alireza Rezazadeh. 2013. Artificial neural network training using a new efficient optimization algorithm. Applied Soft Computing 13, 2 (2013), 1206--1213.
[5]
Arthur Asuncion and David Newman. 2007. UCI machine learning repository.
[6]
Vincenzo Bianco, Oronzio Manca, and Sergio Nardini. 2009. Electricity consumption forecasting in Italy using linear regression models. Energy 34, 9 (2009), 1413--1421.
[7]
Christian Blum and Andrea Roli. 2003. Metaheuristics in Combinatorial Optimization: Overview and Conceptual Comparison. ACM Computing Surveys 35, 3 (2003), 268--308.
[8]
Ervin Ceperic, Vladimir Ceperic, and Adrijan Baric. 2013. A strategy for short-term load forecasting by support vector regression machines. IEEE Transactions on Power Systems 28, 4 (2013), 4356--4364.
[9]
Javier Contreras, Rosario Espinola, Francisco J Nogales, and Antonio J Conejo. 2003. ARIMA models to predict next-day electricity prices. IEEE Transactions on Power Systems 18, 3 (2003), 1014--1020.
[10]
Christian Darken, Joseph Chang, and John Moody. 1992. Learning rate schedules for faster stochastic gradient search. In Proceedings of the Neural Networks for Signal Processing. 3--12.
[11]
Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio. 2014. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In Proceedings of the Neural Information Processing Systems. 2933--2941.
[12]
John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, 7 (2011), 2121--2159.
[13]
Xi Fang, Satyajayant Misra, Guoliang Xue, and Dejun Yang. 2011. Smart grid---The new and improved power grid: A survey. IEEE Communications Surveys and Tutorials 14, 4 (2011), 944--980.
[14]
Eugene A Feinberg and Dora Genethliou. 2005. Load forecasting. In Proceedings of the Applied Mathematics for Restructured Electric Power Systems. 269--285.
[15]
Fred Glover and Gary A. Kochenberger. 1993. Handbook of Metaheuristics. Kluwer Academic Publishers, Boston, MA.
[16]
Luis Hernandez, Carlos Baladrón, Javier M Aguiar, Belén Carro, Antonio J Sanchez-Esguevillas, and Jaime Lloret. 2013. Short-term load forecasting for microgrids based on artificial neural networks. Energies 6, 3 (2013), 1385--1408.
[17]
Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. 2012. Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf
[18]
Abdollah Kavousi-Fard, Haidar Samet, and Fatemeh Marzbani. 2014. A new hybrid modified firefly algorithm and support vector regression model for accurate short term load forecasting. Expert Systems with Applications 41, 13 (2014), 6047--6056.
[19]
James Kennedy and Russell Eberhart. 1995. Particle swarm optimization. In Proceedings of the Neural Networks, Vol. 4. 1942--1948.
[20]
Jack Kiefer and Jacob Wolfowitz. 1952. Stochastic estimation of the maximum of a regression function. Mathematical Statistics 23, 3 (1952), 462--466.
[21]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. https://arxiv.org/abs/1412.6980 arXiv:1412.6980.
[22]
Weicong Kong, Zhao Yang Dong, Youwei Jia, David J Hill, Yan Xu, and Yuan Zhang. 2017. Short-term residential load forecasting based on LSTM recurrent neural network. IEEE Transactions on Smart Grid 10, 1 (2017), 841--851.
[23]
Daniel L Marino, Kasun Amarasinghe, and Milos Manic. 2016. Building energy load forecasting using deep neural networks. In Proceedings of the IEEE Industrial Electronics Society. 7046--7051.
[24]
Dong C Park, MA El-Sharkawi, RJ Marks, LE Atlas, and MJ Damborg. 1991. Electric load forecasting using an artificial neural network. IEEE Transactions on Power Systems 6, 2 (1991), 442--449.
[25]
Ning Qian. 1999. On the momentum term in gradient descent learning algorithms. Neural networks 12, 1 (1999), 145--151.
[26]
Muhammad Qamar Raza and Zuhairi Baharudin. 2012. A review on short term load forecasting using hybrid neural network techniques. In Proceedings of the IEEE on Power and Energy. 846--851.
[27]
Injong Rhee, Minsu Shin, Seongik Hong, Kyunghan Lee, Seong Joon Kim, and Song Chong. 2011. On the levy-walk nature of human mobility. IEEE/ACM Transactions on Networking 19, 3 (2011), 630--643.
[28]
Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. https://arxiv.org/abs/1609.04747 arXiv:1609.04747.
[29]
Seunghyoung Ryu, Jaekoo Noh, and Hongseok Kim. 2017. Deep neural network based demand side short term load forecasting. Energies 10, 1 (2017), 3.
[30]
Chun-Wei Tsai, Alexander Pelov, Ming-Chao Chiang, Chu-Sing Yang, and Tzung-Pei Hong. 2014. Computational awareness for smart grid: A review. International Journal of Machine Learning and Cybernetics 5, 1 (2014), 151--163.
[31]
Ehsan Valian, Shahram Mohanna, and Saeed Tavakoli. 2011. Improved cuckoo search algorithm for feedforward neural network training. International Journal of Artificial Intelligence and Applications 2, 3 (2011), 36--43.
[32]
Gai-Ge Wang, Amir Hossein Gandomi, Xin-She Yang, and Amir Hossein Alavi. 2014. A novel improved accelerated particle swarm optimization algorithm for global numerical optimization. Engineering Computations 31, 7 (2014), 1198--1220.
[33]
Lin Wang, Yi Zeng, and Tao Chen. 2015. Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Systems with Applications 42, 2 (2015), 855--863.
[34]
Xin-She Yang. 2010. Nature-inspired metaheuristic algorithms. Luniver press Publishers, Beckington, Frome.
[35]
Xin-She Yang and Suash Deb. 2009. Cuckoo search via Lévy flights. In Proceedings of the Nature and Biologically Inspired Computing. 210--214.
[36]
Jing-Ru Zhang, Jun Zhang, Tat-Ming Lok, and Michael R Lyu. 2007. A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training. Applied Mathematics and Computation 185, 2 (2007), 1026--1037.

Index Terms

  1. An Effective Optimizer based on Global and Local Searched Experiences for Short-term Electricity Consumption Forecasting
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          ACM ICEA '20: Proceedings of the 2020 ACM International Conference on Intelligent Computing and its Emerging Applications
          December 2020
          219 pages
          ISBN:9781450383042
          DOI:10.1145/3440943
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 27 September 2021

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Short-term load forecasting
          2. and deep neural network
          3. optimizer

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Funding Sources

          • MOST108-2221-E-005-021-MY3

          Conference

          ACM ICEA '20
          Sponsor:

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • 0
            Total Citations
          • 31
            Total Downloads
          • Downloads (Last 12 months)2
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 15 Feb 2025

          Other Metrics

          Citations

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media