skip to main content
10.1145/3440943.3444729acmconferencesArticle/Chapter ViewAbstractPublication PagesiceaConference Proceedingsconference-collections
research-article

The Whale Optimization Algorithm for Hyperparameter Optimization in Network-Wide Traffic Speed Prediction

Published: 27 September 2021 Publication History

Abstract

Since there are way too many possible combinations of hyperparameters for training a desired deep neural network (DNN) model, finding out a set of suitable values for them is typically a difficult topic for the researchers when they use DNN for solving forecasting problems. In addition to manual tuning and trial-and-error for hyperparameters, how to automatically determine the values of hyperparameters has become a critical problem in recent years. In this study, we present a metaheuristic algorithm based on the whale optimization algorithm (WOA) to select suitable hyperparameters for the DNN because WOA demonstrates brilliant convergence speed in many optimization problems and the local optima avoidance mechanism is devised to prevent the searches from trapping into suboptimal solution easily. To validate the feasibility of the proposed algorithm, we compared it with several state-of-the-art hyperparameter selection algorithms for DNN in solving the network-wide traffic speed prediction problem. The experimental results show that WOA not only behaves much more stable but also outperforms all the other hyperparameter selection algorithms compared in this study in terms of the mean square error, mean average error, and mean average percentage error.

References

[1]
Ernesto P. Adorio. 2005. MVF - Multivariate Test Functions Library in C for Unconstrained Global Optimization. Technical Report. University of the Philippines, Diliman.
[2]
James S. Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for hyper-parameter optimization. In Proceedings of Advances in Neural Information Processing Systems. 2546--2554.
[3]
George E. P. Box, Gwilym M. Jenkins, Gregory C. Reinsel, and Greta M. Ljung. 2015. Time series analysis: Forecasting and control. John Wiley & Sons.
[4]
Zhiyong Cui, Ruimin Ke, Ziyuan Pu, and Yinhai Wang. 2018. Deep bidirectional and unidirectional LSTM recurrent neural network for network-wide traffic speed prediction. arXiv preprint arXiv:1801.02143 (2018).
[5]
Stefan Falkner, Aaron Klein, and Frank Hutter. 2018. BOHB: Robust and efficient hyperparameter optimization at scale. arXiv preprint arXiv:1807.01774 (2018).
[6]
Rui Fu, Zuo Zhang, and Li Li. 2016. Using LSTM and GRU neural network methods for traffic flow prediction. In Proceedings of Youth Academic Annual Conference of Chinese Association of Automation. 324--328.
[7]
Geoffrey E. Hinton, Simon Osindero, and Yee-Whye Teh. 2006. A fast learning algorithm for deep belief nets. Neural Computation 18, 7 (2006), 1527--1554.
[8]
Wenhao Huang, Guojie Song, Haikun Hong, and Kunqing Xie. 2014. Deep architecture for traffic flow prediction: Deep belief networks with multitask learning. IEEE Transactions on Intelligent Transportation Systems 15, 5 (2014), 2191--2201.
[9]
Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2011. Sequential model-based optimization for general algorithm configuration. In Proceedings of International Conference on Learning and Intelligent Optimization. 507--523.
[10]
Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Technical Report. University of Toronto.
[11]
Takashi Kuremoto, Shinsuke Kimura, Kunikazu Kobayashi, and Masanao Obayashi. 2014. Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing 137 (2014), 47--56.
[12]
Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra, and Yoshua Bengio. 2007. An empirical evaluation of deep architectures on problems with many factors of variation. In Proceedings of International Conference on Machine Learning. 473--480.
[13]
Yann LeCun, Leon Bottou, G. Orr, and Klaus-Robert Muller. 1998. Efficient backprop. Neural Networks: Tricks of the Trade (1998), 9--48.
[14]
Yann LeCun, Corinna Cortes, and Chris Burges. 2010. MNIST Handwritten Digit Database. http://yann.lecun.com/exdb/mnist/
[15]
Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2017. Hyperband: A novel bandit-based approach to hyperparameter optimization. The Journal of Machine Learning Research 18, 1 (2017), 6765--6816.
[16]
Pablo Ribalta Lorenzo, Jakub Nalepa, Michal Kawulok, Luciano Sanchez Ramos, and José Ranilla Pastor. 2017. Particle swarm optimization for hyper-parameter selection in deep neural networks. In Proceedings of Genetic and Evolutionary Computation Conference. 481--488.
[17]
Ilya Loshchilov and Frank Hutter.2016. CMA-ES for hyperparameter optimization of deep neural networks. arXiv preprint arXiv:1604.07269 (2016).
[18]
Xiaolei Ma, Zhuang Dai, Zhengbing He, Jihui Ma, Yong Wang, and Yunpeng Wang. 2017. Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction. Sensors 17, 4 (2017), 818.
[19]
Seyedali Mirjalili and Andrew Lewis. 2016. The whale optimization algorithm. Advances in Engineering Software 95 (2016), 51--67.
[20]
Iwao Okutani and Yorgos J Stephanedes. 1984. Dynamic prediction of traffic volume through Kalman filtering theory. Transportation Research Part B: Methodological 18 (1984), 1--11.
[21]
Lu Peng, Shan Liu, Rui Liu, and Lin Wang. 2018. Effective long short-term memory with differential evolution algorithm for electricity price prediction. Energy 162 (2018), 1301--1314.
[22]
Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Proceedings of Advances in Neural Information Processing Systems. 2951--2959.
[23]
Mascha Van Der Voort, Mark Dougherty, and Susan Watson. 1996. Combining Kohonen maps with ARIMA time series models to forecast traffic flow. Transportation Research Part C: Emerging Technologies 4, 5 (1996), 307--318.
[24]
Billy M. Williams and Lester A. Hoel. 2003. Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: Theoretical basis and empirical results. Journal of Transportation Engineering 129, 6 (2003), 664--672.
[25]
Chun-Hsin Wu, Jan-Ming Ho, and Der-Tsai Lee. 2004. Travel-time prediction with support vector regression. IEEE Transactions on Intelligent Transportation Systems 5, 4 (2004), 276--281.
[26]
G. Peter Zhang. 2003. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50 (2003), 159--175.
[27]
Weibin Zhang, Yinghao Yu, Yong Qi, Feng Shu, and Yinhai Wang. 2019. Short-term traffic flow prediction based on spatio-temporal analysis and CNN deep learning. Transportmetrica A: Transport Science 15, 2 (2019), 1688--1711.
[28]
Yunlong Zhang and Zhirui Ye. 2008. Short-term traffic flow forecasting using fuzzy logic system methods. Journal of Intelligent Transportation Systems 12, 3 (2008), 102--112.
[29]
Zheng Zhao, Weihai Chen, Xingming Wu, Peter C. Y. Chen, and Jingmeng Liu. 2017. LSTM network: A deep learning approach for short-term traffic forecast. IET Intelligent Transport Systems 11, 2 (2017), 68--75.

Index Terms

  1. The Whale Optimization Algorithm for Hyperparameter Optimization in Network-Wide Traffic Speed Prediction
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          ACM ICEA '20: Proceedings of the 2020 ACM International Conference on Intelligent Computing and its Emerging Applications
          December 2020
          219 pages
          ISBN:9781450383042
          DOI:10.1145/3440943
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 27 September 2021

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Whale optimization algorithm
          2. and deep neural network
          3. hyperparameter optimization

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Funding Sources

          Conference

          ACM ICEA '20
          Sponsor:

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • 0
            Total Citations
          • 71
            Total Downloads
          • Downloads (Last 12 months)9
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 15 Feb 2025

          Other Metrics

          Citations

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media