poster

ONE-NAS: an online neuroevolution based neural architecture search for time series forecasting

Authors:

Travis DesellAuthors Info & Claims

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 659 - 662

https://doi.org/10.1145/3520304.3528962

Published: 19 July 2022 Publication History

Abstract

Time series forecasting (TSF) is one of the most important tasks in data science, as accurate time series (TS) predictions can drive and advance a wide variety of domains including finance, transportation, health care, and power systems. However, real-world utilization of machine learning (ML) models for TSF suffers due to data drift. To address this, models must be periodically retained or redesigned, which requires significant human and computational resources. This work presents the Online NeuroEvolution based Neural Architecture Search (ONE-NAS) algorithm, which to the authors' knowledge is the first neural architecture search algorithm capable of automatically designing and training new recurrent neural networks (RNNs) in an online setting. Without any pretraining, ONE-NAS utilizes populations of RNNs which are continuously updated with new network structures and weights in response to new multivariate input data. ONE-NAS is tested on real-world large-scale multivariate wind turbine data and is shown to outperform traditional statistical time series forecasting, including naive, moving average, and exponential smoothing methods.

References

[1]

Sunil Bhatnagar, Vivek Lal, Shiv D Gupta, Om P Gupta, et al. 2012. Forecasting incidence of dengue in Rajasthan, using time series analyses. Indian journal of public health 56, 4 (2012), 281.

[2]

Chris M Bishop. 1995. Training with noise is equivalent to Tikhonov regularization. Neural computation 7, 1 (1995), 108--116.

[3]

Christopher M Bishop et al. 1995. Neural networks for pattern recognition. Oxford university press.

[4]

R. D. Blumofe and C. E. Leiserson. 1994. Scheduling Multithreaded Computations by Work Stealing. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science (FOCS '94). Santa Fe, New Mexico, 356--368.

[5]

Jian Cao, Zhi Li, and Jian Li. 2019. Financial time series forecasting model based on CEEMDAN and LSTM. Physica A: Statistical Mechanics and its Applications 519 (2019), 127--139.

[6]

Chris Chatfield. 2000. Time-series forecasting. Chapman and Hall/CRC.

[7]

Travis Desell, Dave Anderson, Malik Magdon-Ismail, Boleslaw Szymanski Heidi Newberg, and Carlos Varela. 2010. An Analysis of Massively Distributed Evolutionary Algorithms. In The 2010 IEEE Congress on Evolutionary Computation (IEEE CEC 2010). Barcelona, Spain.

[8]

Tonya Fields, George Hsieh, and Jules Chenou. 2019. Mitigating drift in time series data with noise augmentation. In 2019 International Conference on Computational Science and Computational Intelligence (CSCI). IEEE, 227--230.

[9]

Bidisha Ghosh, Biswajit Basu, and Margaret O'Mahony. 2005. Time-series modelling for forecasting vehicular traffic flow in Dublin. In 84th Annual Meeting of the Transportation Research Board, Washington, DC.

[10]

Tian Guo, Zhao Xu, Xin Yao, Haifeng Chen, Karl Aberer, and Koichi Funaya. 2016. Robust online time series prediction with recurrent neural networks. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA). Ieee, 816--825.

[11]

Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. 2015. An empirical exploration of recurrent network architectures. In International Conference on Machine Learning. 2342--2350.

Digital Library

[12]

Jesus L Lobo, Ibai Laña, Javier Del Ser, Miren Nekane Bilbao, and Nikola Kasabov. 2018. Evolving spiking neural networks for online learning over drifting data streams. Neural Networks 108 (2018), 1--19.

[13]

Zimeng Lyu, Joshua Karnas, AbdElRahman ElSaid, Mohamed Mkaouer, and Travis Desell. 2021. Improving Distributed Neuroevolution Using Island Extinction and Repopulation. The 24th International Conference on the Applications of Evolutionary Computation (EvoStar: EvoApps) (2021).

[14]

Zimeng Lyu, Shuchita Patwardhan, David Stadem, James Langfeld, Steve Benson, Seth Thoelke, and Travis Desell. 2021. Neuroevolution of recurrent neural networks for time series forecasting of coal-fired power plant operating parameters. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. 1735--1743.

Digital Library

[15]

Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. 2018. Statistical and Machine Learning forecasting methods: Concerns and ways forward. PloS one 13, 3 (2018), e0194889.

[16]

Alexander Ororbia, AbdElRahman ElSaid, and Travis Desell. 2019. Investigating Recurrent Neural Network Memory Structures Using Neuro-evolution. In Proceedings of the Genetic and Evolutionary Computation Conference (Prague, Czech Republic) (GECCO '19). ACM, New York, NY, USA, 446--455.

Digital Library

[17]

EN Osegi. 2020. Using the hierarchical temporal memory spatial pooler for short-term forecasting of electrical load time series. Applied Computing and Informatics (2020).

[18]

Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In International Conference on Machine Learning. 1310--1318.

Digital Library

[19]

Kenneth Stanley and Risto Miikkulainen. 2002. Evolving neural networks through augmenting topologies. Evolutionary computation 10, 2 (2002), 99--127.

[20]

Shuo Wang, Leandro L Minku, and Xin Yao. 2018. A systematic study of online class imbalance learning with concept drift. IEEE transactions on neural networks and learning systems 29, 10 (2018), 4802--4821.

Cited By

Fischer RSaadallah ABaeza-Yates RBonchi F(2024)AutoXPCR: Automated Multi-Objective Model Selection for Time Series ForecastingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672057(806-815)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672057
Wang CChen XWu CWang H(2024)Automatic time series forecasting model design based on pruningApplied Soft Computing10.1016/j.asoc.2024.111804162(111804)Online publication date: Sep-2024
https://doi.org/10.1016/j.asoc.2024.111804
Sadr MZhu YHu P(2023)Multivariate Variance-Based Genetic Ensemble Learning for Satellite Anomaly DetectionIEEE Transactions on Vehicular Technology10.1109/TVT.2023.3285599(1-10)Online publication date: 2023
https://doi.org/10.1109/TVT.2023.3285599
Show More Cited By

Index Terms

ONE-NAS: an online neuroevolution based neural architecture search for time series forecasting

Recommendations

Online evolutionary neural architecture search for multivariate non-stationary time series forecasting
Abstract
Time series forecasting (TSF) is one of the most important tasks in data science. TSF models are usually pre-trained with historical data and then applied on future unseen datapoints. However, real-world time series data is usually non-...
Graphical abstract

Display Omitted
Highlights
- We present ONE-NAS, the first algorithm capable of designing and training RNNs in real-time as data arrives as an online fashion.
Co-evolving Recurrent Neural Networks and their Hyperparameters with Simplex Hyperparameter Optimization
GECCO '23 Companion: Proceedings of the Companion Conference on Genetic and Evolutionary Computation

Designing machine learning models involves determining not only the network architecture, but also non-architectural elements such as training hyperparameters. Further confounding this problem, different architectures and datasets will perform more ...
Time series forecasting by recurrent product unit neural networks

Time series forecasting (TSF) consists on estimating models to predict future values based on previously observed values of time series, and it can be applied to solve many real-world problems. TSF has been traditionally tackled by considering ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2022

2395 pages

ISBN:9781450392686

DOI:10.1145/3520304

Editor:
Jonathan E. Fieldsend
University of Exeter
,
General Chair:
Markus Wagner
The University of Adelaide

Copyright © 2022 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2022

Check for updates

Author Tags

Qualifiers

Poster

Conference

GECCO '22

Sponsor:

SIGEVO

GECCO '22: Genetic and Evolutionary Computation Conference

July 9 - 13, 2022

Massachusetts, Boston

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
113
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)4

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fischer RSaadallah ABaeza-Yates RBonchi F(2024)AutoXPCR: Automated Multi-Objective Model Selection for Time Series ForecastingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672057(806-815)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672057
Wang CChen XWu CWang H(2024)Automatic time series forecasting model design based on pruningApplied Soft Computing10.1016/j.asoc.2024.111804162(111804)Online publication date: Sep-2024
https://doi.org/10.1016/j.asoc.2024.111804
Sadr MZhu YHu P(2023)Multivariate Variance-Based Genetic Ensemble Learning for Satellite Anomaly DetectionIEEE Transactions on Vehicular Technology10.1109/TVT.2023.3285599(1-10)Online publication date: 2023
https://doi.org/10.1109/TVT.2023.3285599
Sadr MZhu YHu P(2023)Satellite Anomaly Detection Using Variance Based Genetic Ensemble of Neural NetworksICC 2023 - IEEE International Conference on Communications10.1109/ICC45041.2023.10278933(4070-4075)Online publication date: 28-May-2023
https://doi.org/10.1109/ICC45041.2023.10278933

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten