Abstract
Based on quantile long short-term memory (Q-LSTM), we consider the comprehensive utilization of multiple quantiles, proposing a simultaneous estimation version of Q-LSTM, composite quantile regression LSTM (CQR-LSTM). The method simultaneously estimates multiple quantile functions instead of estimating them separately. It makes sense that simultaneous estimation allows multiple quantiles to share strength among them to get better predictions. Furthermore, we also propose a novel approach, noncrossing composite quantile regression LSTM (NCQR-LSTM), to solve the quantile crossing problem. This method uses an indirect way as follows. Instead of estimating multiple quantiles directly, we estimate the intervals between adjacent quantiles. Since the intervals are guaranteed to be positive by using exponential functions, this completely avoids the problem of quantile crossing. Compared with the commonly used constraint methods for solving the quantile crossing problem, this indirect method makes model optimization easier and more suitable for deep learning. Experiments on a real wind speed dataset show that our methods improve the probabilistic prediction performance and reduce the training cost. In addition, our methods are simple to implement and highly scalable.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bang, S., Cho, H., Jhun, M.: Simultaneous estimation for non-crossing multiple quantile regression with right censored data. Stat. Comput. 26(1–2), 131–147 (2016). https://doi.org/10.1007/s11222-014-9482-0
Bassett Jr., G., Koenker, R.: An empirical quantile function for linear models with IID errors. J. Am. Stat. Assoc. 77(378), 407–415 (1982). https://doi.org/10.1080/01621459.1982.10477826
Cannon, A.J.: Quantile regression neural networks: implementation in R and application to precipitation downscaling. Comput. Geosci. 37(9), 1277–1284 (2011). https://doi.org/10.1016/j.cageo.2010.07.005
Cannon, A.J.: Non-crossing nonlinear regression quantiles by monotone composite quantile regression neural network, with application to rainfall extremes. Stochast. Environ. Res. Risk Assess. 32(11), 3207–3225 (2018). https://doi.org/10.31223/osf.io/wg7sn
Constante-Flores, G.E., Illindala, M.S.: Data-driven probabilistic power flow analysis for a distribution system with renewable energy sources using monte carlo simulation. IEEE Trans. Ind. Appl. 55(1), 174–181 (2019). https://doi.org/10.1109/icps.2017.7945118
Gan, D., Wang, Y., Zhang, N., Zhu, W.: Enhancing short-term probabilistic residential load forecasting with quantile long-short-term memory. J. Eng. 2017(14), 2622–2627 (2017). https://doi.org/10.1049/joe.2017.0833
Gers, F.A., Eck, D., Schmidhuber, J.: Applying LSTM to time series predictable through time-window approaches. In: Tagliaferri, R., Marinaro, M. (eds.) Neural Nets WIRN Vietri-01, pp. 193–200. Springer, London (2002). https://doi.org/10.1007/978-1-4471-0219-9_20
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Huang, Z., Xu, W., Yu, K.: Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)
Jeong, M.C., Lee, S.J., Cha, K., Zi, G., Kong, J.S.: Probabilistic model forecasting for rail wear in seoul metro based on bayesian theory. Eng. Fail. Anal. 96, 202–210 (2019). https://doi.org/10.1016/j.engfailanal.2018.10.001
Koenker, R., Hallock, K.F.: Quantile regression. J. Econ. Perspect. 15(4), 143–156 (2001). https://doi.org/10.1257/jep.15.4.143
Liu, Y., Wu, Y.: Simultaneous multiple non-crossing quantile regression estimation using kernel constraints. J. Nonparametric Stat. 23(2), 415–437 (2011). https://doi.org/10.1080/10485252.2010.537336
Malhotra, P., Vig, L., Shroff, G., Agarwal, P.: Long short term memory networks for anomaly detection in time series. In: Proceedings, p. 89. Presses universitaires de Louvain (2015)
Takeuchi, I., Le, Q.V., Sears, T.D., Smola, A.J.: Nonparametric quantile estimation. J. Mach. Learn. Res. 7(Jul), 1231–1264 (2006)
Tieleman, T., Hinton, G.: Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2), 26–31 (2012)
Xu, Q., Deng, K., Jiang, C., Sun, F., Huang, X.: Composite quantile regression neural network with applications. Expert Syst. Appl. 76, 129–139 (2017). https://doi.org/10.1016/j.eswa.2017.01.054
Zongxia, X., Yong, X., Qinghua, H.: Uncertain data classification with additive kernel support vector machine. Data Knowl. Eng. https://doi.org/10.1016/j.datak.2018.07.004
Acknowledgments
This work is supported by the National Natural Science Foundation of China under Grants 61432011, U1435212, 61105054, and Open Research Program of Key Laboratory of Solar Activity.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, Z., Wen, H. (2019). Composite Quantile Regression Long Short-Term Memory Network. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Text and Time Series. ICANN 2019. Lecture Notes in Computer Science(), vol 11730. Springer, Cham. https://doi.org/10.1007/978-3-030-30490-4_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-30490-4_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30489-8
Online ISBN: 978-3-030-30490-4
eBook Packages: Computer ScienceComputer Science (R0)