Generative AI for Energy: Multi-Horizon Power Consumption Forecasting using Large Language Models

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum Learning. In Proceedings of the 26th Annual International Conference on Machine Learning (Montreal, Quebec, Canada) (ICML 09). Association for Computing Machinery, New York, NY, USA, 41--48. https://doi.org/10.1145/1553374.1553380

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877--1901. https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

Shobhit Chaturvedi, Elangovan Rajasekar, Sukumar Natarajan, and Nick McCullen. 2022. A comparative assessment of SARIMA, LSTM RNN and Fb Prophet models to forecast total and peak monthly energy demand for India. Energy Policy, Vol. 168 (2022), 113097. https://doi.org/10.1016/j.enpol.2022.113097

Alice Chuang. 1991. Time Series Analysis: Univariate and Multivariate Methods. Technometrics, Vol. 33, 1 (1991), 108--109. https://doi.org/10.1080/00401706.1991.10484777

Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).

Vincenzo Della Mea, Mihai Horia Popescu, and Kevin Roitero. 2020. Underlying cause of death identification from death certificates using reverse coding to text and a NLP based deep learning approach. Informatics in Medicine Unlocked, Vol. 21 (2020), 100456.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

Rahul Dey and Fathi M Salem. 2017. Gate-variants of gated recurrent unit (GRU) neural networks. In 2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS). IEEE, 1597--1600.

Everette S Gardner Jr. 1985. Exponential smoothing: The state of the art. Journal of forecasting, Vol. 4, 1 (1985), 1--28.

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

Mehdi Khashei, Mehdi Bijari, and Seyed Reza Hejazi. 2012. Combining seasonal ARIMA models with computational intelligence techniques for time series forecasting. Soft computing, Vol. 16 (2012), 1091--1105.

Ashutosh Kumar Dubey, Abhishek Kumar, Vicente García-Díaz, Arpit Kumar Sharma, and Kishan Kanhaiya. 2021. Study and analysis of SARIMA and LSTM in forecasting time series data. Sustainable Energy Technologies and Assessments, Vol. 47 (2021), 101474. https://doi.org/10.1016/j.seta.2021.101474

Bryan Lim, Sercan Ö Arik, Nicolas Loeff, and Tomas Pfister. 2021. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, Vol. 37, 4 (2021), 1748--1764.

Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024. Large language models: A survey. arXiv preprint arXiv:2402.06196 (2024).

Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. 2019. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437 (2019).

Adhistya Erna Permanasari, Indriana Hidayah, and Isna Alfi Bustoni. 2013. SARIMA (Seasonal ARIMA) implementation on time series to forecast the number of Malaria incidence. In 2013 International Conference on Information Technology and Electrical Engineering (ICITEE). 203--207. https://doi.org/10.1109/ICITEED.2013.6676239

Mihai Horia Popescu, Kevin Roitero, Stefano Travasci, and Vincenzo Della Mea. 2021. Automatic Assignment of ICD-10 Codes to Diagnostic Texts using Transformers Based Techniques. In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI). IEEE, 188--192.

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. (2018).

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., Vol. 21, 140 (2020), 1--67.

Kevin Roitero, Cristina Gattazzo, Andrea Zancola, Vincenzo Della Mea, Stefano Mizzaro, et al. 2022. Causal Text-to-Text Transformers for Water Pollution Forecasting. In CEUR WORKSHOP PROCEEDINGS, Vol. 3463. CEUR-WS.

Kevin Roitero, Beatrice Portelli, Mihai Horia Popescu, and Vincenzo Della Mea. 2021. DiLBERT: Cheap Embeddings for Disease Related Medical NLP. IEEE Access, Vol. 9 (2021), 159714--159723.

Kevin Roitero, Beatrice Portelli, Giuseppe Serra, Vincenzo Della Mea, Stefano Mizzaro, Gianni Cerro, Michele Vitelli, and Mario Molinara. 2023. Detection of Wastewater Pollution Through Natural Language Generation With a Low-Cost Sensing Platform. IEEE Access, Vol. 11 (2023), 50272--50284. https://doi.org/10.1109/ACCESS.2023.3277535

David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. 2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, Vol. 36, 3 (2020), 1181--1191.

Robert H. Shumway and David S. Stoffer. 2017. ARIMA Models. Springer International Publishing, Cham, 75--163. https://doi.org/10.1007/978--3--319--52452--8_3

Shixiong Wang, Chongshou Li, and Andrew Lim. 2019. Why are the ARIMA and SARIMA not sufficient. arXiv preprint arXiv:1904.07632 (2019).

Hao Xue, Flora D Salim, Yongli Ren, and Charles LA Clarke. 2022. Translating Human Mobility Forecasting through Natural Language Generation. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 1224--1233.

Hao Xue, Tianye Tang, Ali Payani, and Flora D Salim. 2024. Prompt mining for language-based human mobility forecasting. arXiv preprint arXiv:2403.03544 (2024).

Hao Xue, Bhanu Prakash Voutharoj, and Flora D Salim. 2022. Leveraging Language Foundation Models for Human Mobility Forecasting. arXiv preprint arXiv:2209.05479 (2022).

Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang. 2019. A review of recurrent neural networks: LSTM cells and network architectures. Neural computation, Vol. 31, 7 (2019), 1235--1270.