Abstract
Quantile regression neural network (QRNN) model has received increasing attention in various fields to provide conditional quantiles of responses. However, almost all the available literature about QRNN is devoted to handling the case with one-dimensional responses, which presents a great limitation when we focus on the quantiles of multivariate responses. To deal with this issue, we propose a novel multiple-output quantile regression neural network (MOQRNN) model in this paper to estimate the conditional quantiles of multivariate data. The MOQRNN model is constructed by the following steps. Step 1 acquires the conditional distribution of multivariate responses by a nonparametric method. Step 2 obtains the optimal transport map that pushes the spherical uniform distribution forward to the conditional distribution through the input convex neural network (ICNN). Step 3 provides the conditional quantile contours and regions by the ICNN-based optimal transport map. In both simulation studies and real data application, comparative analyses with the existing method demonstrate that the proposed MOQRNN model is more appealing to yield excellent quantile contours, which are not only smoother but also closer to their theoretical counterparts.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abiodun, O.I., Jantan, A., Omolara, A.E., Dada, K.V., Mohamed, N.A., Arshad, H.: State-of-the-art in artificial neural network applications: a survey. Heliyon 4(11), e00938 (2018). https://doi.org/10.1016/j.heliyon.2018.e00938
Amos, B., Xu, L., Kolter, J.Z.: Input convex neural networks. In: International Conference on Machine Learning, PMLR, pp. 146-155 (2017). https://proceedings.mlr.press/v70/amos17b.html
Brenier, Y.: Polar factorization and monotone rearrangement of vector-valued functions. Commun. Pur. Appl. Math. 44(4), 375–417 (1991). https://doi.org/10.1002/cpa.3160440402
Cannon, A.J.: Quantile regression neural networks: implementation in R and application to precipitation downscaling. Comput. Geosci. 37(9), 1277–1284 (2011). https://doi.org/10.1016/j.cageo.2010.07.005
Chakraborty, A., Chaudhuri, P.: The spatial distribution in infinite dimensional spaces and related quantiles and depths. Ann. Stat. 42(3), 1203–1231 (2014). https://doi.org/10.1214/14-AOS1226
Chakraborty, A., Chaudhuri, P.: Tests for high-dimensional data based on means, spatial signs and spatial ranks. Ann. Stat. 45(2), 771–799 (2017). https://doi.org/10.1214/16-AOS1467
Chaudhuri, P., Sengupta, D.: Sign tests in multidimension: inference based on the geometry of the data cloud. J. Am. Stat. Assoc. 88(424), 1363–1370 (1993). https://doi.org/10.1080/01621459.1993.10476419
Chaudhuri, P.: On a geometric notion of quantiles for multivariate data. J. Am. Stat. Assoc. 91(434), 862–872 (1996). https://doi.org/10.1080/01621459.1996.10476954
Cheng, Y., De Gooijer, J.G.: On the uth geometric conditional quantile. J. Stat. Plan. Infer. 137(6), 1914–1930 (2007). https://doi.org/10.1016/j.jspi.2006.02.014
Chernozhukov, V., Galichon, A., Hallin, M., Henry, M.: Monge-Kantorovich depth, quantiles, ranks and signs. 45(1), 223–256 (2017). https://doi.org/10.1214/16-AOS1450
Dane, S.: CalCOFI dataset. https://www.kaggle.com/datasets/sohier/calcofi (2017). Accessed 30 May 2023
del Barrio, E., Cuesta-Albertos, J.A., Matr\(\acute{\rm {a}}\)n, C., Mayo-\(\acute{\rm {I}}\)scar A.: Robust clustering tools based on optimal transportation Stat. Comput. 29, 139–160 (2019). https://doi.org/10.1007/s11222-018-9800-z
del Barrio, E., Sanz, A.G., Hallin, M.: Nonparametric multiple-output center-outward quantile regression (2022). arXiv: 2204.11756
Felix, M., La Vecchia, D., Liu, H., Ma, Y.: Some novel aspects of quantile regression: local stationarity, random forests and optimal transportation (2023). arXiv: 2303.11054
Gonz\(\acute{a}\)lez-Sanz, A., De Lara, L., B\(\acute{\rm {e}}\)thune, L., Loubes, J.M.: Gan estimation of lipschitz optimal transport maps (2022). arXiv:2202.07965
Hallin, M., Paindaveine, D., \(\check{\rm {S}}\)iman, M., Wei, Y., Serfling, R., Zuo, Y., Mizera, I.: MUltivariate quantiles and multiple-output regression quantiles: from l1 optimization to halfspace depth [with Discussion and Rejoinder]. Ann. Stat. 38(2), 635–703 (2010). https://www.jstor.org/stable/25662257
Hallin, M., Lu, Z., Paindaveine, D., \(\check{\rm {S}}\)iman, M.: Local bilinear multiple-output quantile/depth regression. Bernoulli 21(3), 1435–1466 (2015). https://doi.org/10.3150/14-BEJ610
Hallin, M., \(\check{\rm {S}}\)iman, M.: Elliptical multiple-output quantile regression and convex optimization. Stat. Probabil. Lett. 109, 232–237 (2016). https://doi.org/10.1016/j.spl.2015.11.021
Hallin, M., Del Barrio, E., Cuesta-Albertos, J., Matrán, C.: Distribution and quantile functions, ranks and signs in dimension d: a measure transportation approach. Ann. Stat. 49(2), 1139–1165 (2021). https://doi.org/10.1214/20-AOS1996
He, Y., Xu, Q., Wan, J., Yang, S.: Short-term power load probability density forecasting based on quantile regression neural network and triangle kernel function. Energy 114, 498–512 (2016). https://doi.org/10.1016/j.energy.2016.08.023
He, Y., Zhang, W.: Probability density forecasting of wind power based on multi-core parallel quantile regression neural network. Knowl. Based Syst. 209, 106431 (2020). https://doi.org/10.1016/j.knosys.2020.106431
Hlubinka, D., \(\check{\rm {S}}\)iman, M. J.: On elliptical quantiles in the quantile regression setup. Multivariate Anal. 116, 163–171 (2013). https://doi.org/10.1016/j.jmva.2012.11.016
Hlubinka, D., \(\check{\rm {S}}\)iman, M.: On generalized elliptical quantiles in the nonlinear quantile regression setup. Test Spain 24(2), 249–264 (2015). https://doi.org/10.1007/s11749-014-0405-3
Hu, Y., Gramacy, R.B., Lian, H.: Bayesian quantile regression for single-index models. Stat. Comput. 23, 437–454 (2013). https://doi.org/10.1007/s11222-012-9321-0
Hu, Y., Zhao, K., Lian, H.: Bayesian quantile regression for partially linear additive models. Stat. Comput. 25, 651–668 (2015). https://doi.org/10.1007/s11222-013-9446-9
Keilbar, G., Wang, W.: Modelling systemic risk using neural network quantile regression. Empir. Econ. 62(1), 93–118 (2022). https://doi.org/10.1007/s00181-021-02035-1
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv: 1412.6980
Koenker, R., Bassett, G., Jr.: Regression quantiles. Econometrica 46(1), 33–50 (1978). https://doi.org/10.2307/1913643
Lin, J.Y., Guo, S., Xie, L., Du, R., Xu, G.: A discrete method for the initialization of semi-discrete optimal transport problem. Knowl. Based. Syst. 212, 106608 (2021). https://doi.org/10.1016/j.knosys.2020.106608
Makkuva, A., Taghvaei, A., Oh, S., Lee J.: Optimal transport mapping via input convex neural networks. In: International Conference on Machine Learning, PMLR, pp. 6672–6681 (2020). https://proceedings.mlr.press/v119/makkuva20a.html
McCann, R.J.: Existence and uniqueness of monotone measure-preserving maps. Duke Math. J. 80(2), 309–323 (1995). https://doi.org/10.1215/S0012-7094-95-08013-2
Meinshausen, N., Ridgeway, G.: Quantile regression forests. J. Mach. Learn. Res. 7, 983–999 (2006)
Monge, G.: M\(\acute{\rm {e}}\)moire sur la th\(\acute{\rm {e}}\)orie des d\(\acute{\rm {e}}\)blais et des remblais. Mem. Math. Phys. Acad. Royale Sci. 666–704 (1781). https://cir.nii.ac.jp/crid/1572261550791499008
Naeem, M.A., Karim, S., Tiwari, A.K.: Quantifying systemic risk in US industries using neural network quantile regression. Res. Int. Bus. Finance 61, 101648 (2022). https://doi.org/10.1016/j.ribaf.2022.101648
Nordhausen, K., Oja, H., Tyler, D.E.: On the efficiency of invariant multivariate sign and rank tests. In: Liski, E.P., Isotalo, J., Niemel\(\ddot{a}\), J., Puntanen, S., Styan, G.P.H. (eds.) Festschrift for Tarmo Pukkila on his 60th Birthday, pp. 217–231. University of Tampere, Tampere (2006)
Paindaveine, D., \(\check{\rm {S}}\)iman, M.: Computing multiple-output regression quantile regions. Comput. Stat. Data An. 56(4), 840–853 (2012). https://doi.org/10.1016/j.csda.2010.11.014
Peyr\(\acute{\rm {e}}\), G., Cuturi, M.: Computational optimal transport: With applications to data science. Found. Trends Mach. Le. 11(5–6), 355–607 (2019). https://doi.org/10.1561/2200000073
Pradeepkumar, D., Ravi, V.: Forecasting financial time series volatility using particle swarm optimization trained quantile regression neural network. Appl. Soft Comput. 58, 35–52 (2017). https://doi.org/10.1016/j.asoc.2017.04.014
Santos, B., Kneib, T.: Noncrossing structured additive multiple-output Bayesian quantile regression models. Stat. Comput. 30, 855–869 (2020). https://doi.org/10.1007/s11222-020-09925-x
Segers, J., Van den Akker, R., Werker, B.J.: Semiparametric Gaussian copula models: geometry and efficient rank-based estimation. Ann. Stat. 42(5), 1911–1940 (2014). https://doi.org/10.1214/14-AOS1244
Serfling, R.: Quantile functions for multivariate analysis: approaches and applications. Stat. Neerl. 56(2), 214–232 (2002). https://doi.org/10.1111/1467-9574.00195
Takeuchi, I., Le, Q., Sears, T., Smola, A.: Nonparametric quantile estimation. J. Mach. Learn. Res. 7, 1231–1264 (2006)
Taylor, J.W.: A quantile regression neural network approach to estimating the conditional density of multiperiod returns. J. Forecast. 19(4), 299–311 (2000). https://doi.org/10.1002/1099-131X(200007)19:4<299::AID-FOR775>3.0.CO;2-V
Villani, C.: Topics in Optimal Transportation (Graduate Studies in Mathematics 58). American Mathematical Society, Providence (2003)
White, H.: Nonparametric estimation of conditional quantiles using neural networks. In: Page, C., LePage, R. (eds.) Computing Science and Statistics, pp. 190–199. Springer, New York (1992). https://doi.org/10.1007/978-1-4612-2856-1_25
Wu, T.Z., Yu, K., Yu, Y.: Single-index quantile regression. J. Multivar. Anal. 101(7), 1607–1621 (2010). https://doi.org/10.1016/j.jmva.2010.02.003
Yu, K., Jones, M.: Local linear quantile regression. J. Am. Stat. Assoc. 93(441), 228–237 (1998). https://doi.org/10.1080/01621459.1998.10474104
Zuo, Y., Serfling, R.: General notions of statistical depth function. Ann. Stat. 28(2), 461–482 (2000)
Acknowledgements
This research is funded by the Natural Science Foundation of Zhejiang Province (No. LY22A010006), the National Natural Science Foundation of China (No. U23A2064), the National Social Science Foundation of China (No.23BTJ036), the Fundamental Research Funds for the Provincial University of Zhejiang, the Characteristic & Preponderant Discipline of Key Construction Universities in Zhejiang Province (Zhejiang Gongshang University-Statistics), and Collaborative Innovation Center of Statistical Data Engineering Technology & Application. Besides, the authors also would like to thank the anonymous reviewers for their constructive feedback, especially the comment on evaluating our method through empirical inclusion probability for Example 3 of simulation studies, which led us to improve our paper significantly.
Author information
Authors and Affiliations
Contributions
Ruiting Hao prepared the codes of this work and wrote the main manuscript text; Xiaorong Yang prepared figures and tables of this work, guided the manuscript and helped the revision.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no financial or proprietary interests in any material discussed in this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hao, R., Yang, X. Multiple-output quantile regression neural network. Stat Comput 34, 89 (2024). https://doi.org/10.1007/s11222-024-10408-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11222-024-10408-6