Skip to main content
SpringerLink
Log in

Sparse regression modeling for short- and long‐term natural gas demand prediction

  • Original Research
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

The multivariate adaptive regression splines (MARS) model is a flexible non-parametric sparse regression algorithm and provides an excellent promise to data fitting through nonlinear basis functions. During the last decades, it is employed in many fields of control design, finance, technology, and science. It can be regarded as an extension of linear models that automatically model interactions and nonlinearities. The least absolute shrinkage and selection operator (LASSO) analysis is a variable selection and shrinkage method to linear regression models. It proposes to construct the subset of explanatory variables which minimizes estimation error to a quantitative dependent variable. LASSO is applied to choose the variables and perform the regularization to improve the interpretability and robustness of the model. In this paper, we examine MARS and LASSO to generate natural gas demand forecasts of residential users for the distribution system operators who need both short- and long-term forecasts. We also compare the performance of MARS and LASSO with a simple multiple-linear regression (LR) commonly used in practice. Our analysis reveals that MARS outperforms LASSO and LR in both the average measures and in the worst-case analysis. For 1 day-ahead forecast, MARS provides a MAPE of around 4.8% while the same figure under LASSO and LR reaches 8.3 and 8.5% respectively. However, as the forecasting horizon increases, we observe that the performance of these proposed methods gets worse and for 1 year-ahead forecast, the MAPE values for MARS, LASSO, and LR are 13.4%, 24.8% and 26.3% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Akpınar, M., & Yumuşak, N. (2013). Forecasting household natural gas consumption with ARIMA model: A case study of removing cycle. In IEEE international conference on application of information and communication technologies; 23–25 October 2013 (pp. 319–324).

  • Aydınalp-Köksal, M., & Uğursal. V. I (2008). Comparison of neural network, conditional demand analysis, and engineering approaches for modeling end-use energy consumption in the residential sector. Applied Energy, 85(4), 271–296.

    Google Scholar 

  • Bai, Y., & Li, C. (2016). Daily natural gas consumption forecasting based on a structure-calibrated support vector regression approach. Energy and Buildings, 127, 571–579.

    Google Scholar 

  • Balalla, D.-T., Nguyen-Huy, T., & Deo, R. (2021). MARS model for prediction of short- and long-term global solar radiation. Predictive Modelling for Energy Management and Power Systems Engineering, 391–436.

  • Ben-Tal, A., & Nemirovski, A. (2002). Robust optimization—Methodology and applications. Mathematical Programming, 92(3), 453–480.

    Google Scholar 

  • Beyca, O.-F., Ervural, C., Tatoglu, B., Gokcin Ozuyar, E., P., & Zaim, S. (2019). Using machine learning tools for forecasting natural gas consumption in the province of Istanbul. Energy Economics, 80, 937–949.

    Google Scholar 

  • Brabec, M., Konar, O., Pelikan, E., & Maly, M. (2008). A nonlinear mixed effects model for the prediction of natural gas consumption by individual customers. International Journal of Forecasting, 24(4), 659–678.

    Google Scholar 

  • Briand, L. C., Freimut, B., & Vollei, F. (2004). Using multiple adaptive regression splines to support decision making in code inspections. Journal of Systems and Software, 73(2), 205–217.

    Google Scholar 

  • Brown, R., Kharouf, P., Feng, X., Piessens, L., & Nestor, D. (1994). Development of feed-forward network models to predict gas consumption. In IEEE international conference on neural networks conference proceedings, 2, 802–805.

  • Chen, Y., Chu, C.-W., Chen, M.-I.-C., & Cook, A.-R. (2018). The utility of LASSO-based models for real time forecasts of endemic infectious diseases: A cross country comparison. Journal of Biomedical Informatics, 81, 16–30.

    Google Scholar 

  • Chen, Y., Zhu, L., Gonder, J., Young, S., & Walkowicz, K. (2017). Data-driven fuel consumption estimation: A multivariate adaptive regression spline approach. Transportation Research Part C: Emerging Technology, 83, 134–145.

    Google Scholar 

  • Chou, S. M., Lee, T. S., Shao, Y. E., & Chen, I. F. (2004). Mining the breast cancer pattern using artificial neural networks and multivariate adaptive regression splines. Expert Systems with Applications, 27(1), 133–142.

    Google Scholar 

  • Crino, S., & Brown, D. E. (2007). Global optimization with multivariate adaptive regression splines. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 37(2), 333–340.

    Google Scholar 

  • De Andrés, J., Lorca, P., de Cos Juez, F. J., & Sánchez-Lasheras, F. (2011). Bankruptcy forecasting: A hybrid approach using Fuzzy c-means clustering and Multivariate Adaptive Regression Splines (MARS). Expert Systems with Applications, 38(3), 1866–1875.

    Google Scholar 

  • Demirel, O. F., Zaim, S., Çalışkan, A., & Özuyar, P. (2012). Forecasting natural gas consumption in Istanbul using neural networks and multivariate time series methods. Turkish Journal of Electrical Engineering & Computer Science, 20(5), 695–711.

  • El-Ghaoui, L. (2003). Robust optimization and applications. IMA Tutorial.

  • Friedman, J. H. (1991). Multivariate adaptive regression splines. Annals of Statistics, 19, 1–67.

    Google Scholar 

  • Gascon, A., & Sanchez-Ubeda, E. F. (2018). Automatic specification of piecewise linear additive models: Application to forecasting natural gas demand. Statistics and Computing, 28(1), 201–217.

    Google Scholar 

  • Görücü, F. B. (2004). Artificial neural network modeling for forecasting gas consumption. Energy Sources, 26(3), 299–307.

    Google Scholar 

  • Gümrah, F., Katırcıoğlu, D., Aykan, Y., Okumuş, S., & Kılınçer, N. (2001). Modeling of gas demand using degree-day concept: Case study for Ankara. Energy Sources, 23(2), 101–114.

  • Hastie, T., Tibshirani, R., & Friedman, J. H. (2001). The element of statistical learning. Springer series in statistics. New York: Springer.

    Google Scholar 

  • Hribar, R., Potočnik, P., Šilc, J., & Papa, G. (2019). A comparison of models for forecasting the residential natural gas demand of an urban area. Energy, 167, 511–522.

    Google Scholar 

  • Huang, J.-C., Tsai, Y.-C., Wu, P.-Y., Lien, Y.-H., Chien, C.-Y., Kuo, C.-F., Hung, J.-F., Chen, S.-C., & Kuo, C.-H. (2020). Predictive modeling of blood pressure during hemodialysis: A comparison of linear model, random forest, support vector regression, XGBoost, LASSO regression and ensemble method. In Computer Methods and Programs in Biomedicine (p. 195).

  • James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction of statistical learning. New York: Springer.

    Google Scholar 

  • Karadede, Y., Ozdemir, G., & Aydemir, E. (2017). Breeder hybrid algorithm approach for natural gas demand forecasting model. Energy.

  • Kızılaslan, R., & Karlık, B. (2008). Comparison neural networks models for short term forecasting of natural gas consumption in Istanbul. In: Applications of digital information and web technologies (ICADIWT), (pp. 448–453).

  • Kriner, M. (2007). Survival Analysis with Multivariate Adaptive Regression Splines, Dissertation. LMU München, Faculty of Mathematics, Computer Science and Statistics.

  • Kuter, S., Weber, G.-W., Özmen, A., & Akyürek, Z. (2014). Modern applied mathematics for alternative modelling of the atmospheric effects on satellite images, Modeling, Optimization, Dynamics and Bioeconomy I, Springer Proceedings in Mathematics & Statistics, 73, 469–485.

  • Kuter, S., Weber, G.-W., Akyürek, Z., & Özmen, A. (2015). Inversion of top of atmospheric reflectance values by conic multivariate adaptive regression splines. Inverse Problems in Science and Engineering (IPSE), 23(4), 651–669.

    Google Scholar 

  • Leathwick, J. R., Rowe, D., Richardson, J., Elith, J., & Hastie, T. (2005). Using multivariate adaptive regression splines to predict the distributions of New Zealand’s freshwater diadromous fish. Freshwater Biology, 50(12), 2034–2052.

    Google Scholar 

  • Lee, T. S., & Chen, I. F. (2005). A two-stage hybrid credit scoring model using artificial neural networks and multivariate adaptive regression splines. Expert Systems with Applications, 28(4), 743–752.

    Google Scholar 

  • Lee, T., Chiu, C., Chou, Y., & Lu, C. (2006). Mining the customer credit using classification and regression tree and multivariate adaptive regression splines. Computational Statistics & Data Analysis, 50, 1113–1130.

    Google Scholar 

  • Liu, L., & Lin, M. (1991). Forecasting residential consumption of natural gas using monthly and quarterly time series. International Journal of Forecasting, 7(1), 3–16.

    Google Scholar 

  • Lu, C. J., Lee, T. S., & Lian, C. M. (2012). Sales forecasting for computer wholesalers: A comparison of multivariate adaptive regression splines and artificial neural networks. Decision Support Systems, 54(1), 584–596.

    Google Scholar 

  • Martinez, D. L., Shih, D. T., Chen, V. C., & Kim, S. B. (2015). A convex version of multivariate adaptive regression splines. Computational Statistics & Data Analysis, 81, 89–106.

    Google Scholar 

  • Miguéis, V. L., Camanho, A., & e Cunha, J. F. (2013). Customer attrition in retailing: An application of multivariate adaptive regression splines. Expert Systems with Applications, 40(16), 6225–6232.

    Google Scholar 

  • Musilek, P., Pelikan, E., Brabec, T., & Simunek, M. (2006). Recurrent neural network-based gating for natural gas load prediction system. In IEEE International Conference on Neural Networks Conference Proceedings (pp. 3736–3741).

  • Minitab. (2017). Software. http://www.minitab.com. Accessed 2017.

  • Nalcaci, G., Özmen, A., & Weber, G.-W. (2014). Long-term load forecasting—Models based on MARS, ANN and LR methods. Central Journal of Operational Research (CEJOR), 27(4), 1033–1049.

    Google Scholar 

  • Montgomery, D. C., & Runger, G. C. (2007). Applied statistics and probability for engineers. New York: Wiley.

    Google Scholar 

  • Özmen, A. (2010). Robust conic quadratic programming applied to quality improvement—A robustification of CMARS. MSc Thesis. Scientific Computing, Institute of Applied Mathematics, Middle East Technical University, Ankara, Turkey.

  • Özmen, A. (2016). Robust optimization of spline models and complex regulatory networks—Theory, methods and application. Contribution to Management Science. New York: Springer.

  • Özmen, A., Batmaz, I., & Weber, G.-W. (2014b). Precipitation modeling by polyhedral RCMARS and comparison with MARS and CMARS. Environmental Modeling & Assessment, 19(5), 425–435.

    Google Scholar 

  • Özmen, A., Kropat, E., & Weber, G.-W. (2014a). Spline regression models for complex multi-modal regulatory networks. Optimization Methods and Software (OMS), 29(3), 515–534.

    Google Scholar 

  • Özmen, A., Kropat, E., & Weber, G.-W. (2017). Robust optimization in spline regression models for multi-model regulatory networks under polyhedral uncertainty. Optimization, 66(12), 2135–2155.

    Google Scholar 

  • Özmen, A., Weber, G.-W., Batmaz, I., & Kropat, E. (2011). RCMARS: Robustification of CMARS with different scenarios under polyhedral uncertainty set. Communications in Nonlinear Science and Numerical Simulation (CNSNS), 16(12), 4780–4787.

    Google Scholar 

  • Özmen, A., & Weber, G.-W. (2014). RMARS: Robustification of multivariate adaptive regression spline under polyhedral uncertainty. Journal of Computational and Applied Mathematics, (CAM), 259, 914–924.

    Google Scholar 

  • Özmen, A., Weber, G.-W., & Kropat, E. (2012). Robustification of conic generalized partial linear models under polyhedral uncertainty. International IFNA-ANS scientific Journal “Problems of Nonlinear Analysis in Engineering Systems”, 2(38), 104–113.

    Google Scholar 

  • Özmen, A., Weber, G.-W., Çavuşoğlu, Z., & Defterli, Ö (2013a). The new robust conic GPLM method with an application to finance: Prediction of credit default. Journal of Global Optimization (JOGO), 56(2), 233–249.

    Google Scholar 

  • Özmen, A., Weber, G.-W., & Karimov, A. (2013b). RCMARS: A new optimization supported tool applied on financial data. Pacific Journal of Optimization (PJO), 9(3), 535–552.

    Google Scholar 

  • Özmen, A., Yılmaz, Y., & Weber, G.-W. (2018). Natural gas consumption forecasting model for residential user. Energy Economics, 70, 357–381.

    Google Scholar 

  • Pilla, V. L., Rosenberger, J. M., Chen, V., Engsuwan, N., & Siddappa, S. (2012). A multivariate adaptive regression splines cutting plane approach for solving a two-stage stochastic programming fleet assignment model. European Journal of Operational Research, 216(1), 162–171.

    Google Scholar 

  • Qiao, W., Yang, Z., Kang, Z., & Pan, Z. (2020). Short-term natural gas consumption prediction based on Volterra adaptive filter and improved whale optimization algorithm, Engineering Applications of Artificial Intelligence, 87.

  • Rui, C., Jian, W., Li, W., Ningjie, Y., & Pengyan, Z. (2009). The forecasting of china natural gas consumption based on genetic algorithm. In Fifth International joint conference on INC, IMS and IDC, NCM ‘09 (pp. 1436–1439).

  • Sailor, D. J., & Muñoz, J. R. (1997). Sensitivity of electricity and natural gas consumption to climate in the U.S.A.—Methodology and results for eight states. Energy, 22(10), 987–998.

    Google Scholar 

  • Salford systems-data mining and predictive analytics software. (2018). MARS software. http://www.salfordsystems.com. Accessed 2018.

  • Sarak, H., & Satman, A. (2003). The degree-day method to estimate the residential heating natural gas consumption in Turkey: A case study. Energy, 28(9), 929–939.

    Google Scholar 

  • Samui, P., & Kurup, P. (2012). Multivariate adaptive regression spline (MARS) and least squares support vector machine (LSSVM) for OCR prediction. Soft Computing, 16(8), 1347–1351.

    Google Scholar 

  • Sen, D., Günay, M. E., & Tunç, K. M. M. (2019). Forecasting annual natural gas consumption using socio-economic indicators for making future policies. Energy, 173, 1106–1118.

    Google Scholar 

  • Sermpinis, G., Tsoukas, S., & Zhang, P. (2018). Modelling market implied ratings using LASSO variable selection techniques. Journal of Empirical Finance, 48, 19–35.

    Google Scholar 

  • Sigauke, C., & Chikobvu, D. (2010). Daily peak electricity load forecasting in South Africa using a multivariate non-parametric regression approach. ORiON: The Journal of ORSSA, 26(2).

  • Soldo, B. (2012). Forecasting natural gas consumption. Applied Energy, 92, 26–37.

    Google Scholar 

  • Thaler, M., Grabec, I., & Poredo, A. (2005). Prediction of energy consumption and risk of excess demand in a distribution system. Physica A: Statistical Mechanics and its Applications, 355(1), 46–53.

    Google Scholar 

  • Taylan, P., Weber, G.-W., & Yerlikaya, F. (2010). A new approach to multivariate adaptive regression spline by using Tikhonov regularization and continuous optimization. TOP, 18(2), 377–395.

    Google Scholar 

  • Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Soceity B, 58(1), 267–288.

  • Toksarı, M. (2010). Predicting the natural gas demand based on economic indicators: Case of Turkey. Energy Sources, Part A: Recovery, Utilization, and Environmental Effects, 32(6), 559–566.

    Google Scholar 

  • Tonkovic, Z., Zekic-Susac, M., & Somolanji, M. (2009). Predicting natural gas consumption by neural networks. Tehnicki Vjesnik, 16(3), 51–61.

    Google Scholar 

  • Tsai, J. C., & Chen, V. C. (2005). Flexible and robust implementations of multivariate adaptive regression splines within a wastewater treatment stochastic dynamic program. Quality and Reliability Engineering International, 21(7), 689–699.

    Google Scholar 

  • Uniejewski, B., Marcjasz, G., & Weron, R. (2019). Understanding intraday electricity markets: Variable selection and very short-term price forecasting using LASSO. International Journal of Forecasting, 35(4), 1533–1547.

    Google Scholar 

  • Vondracek, J., Pelikan, E., Konar, O., Cermakova, J., Eben, K., Maly, M., & Brabec, M. (2008). A statistical model for the estimation of natural gas consumption. Applied Energy, 85(5), 362–370.

    Google Scholar 

  • Weber, G.-W., Çavuşoğlu, Z., & Özmen, A. (2012a). Predicting default probabilities in emerging markets by conic generalized partial linear models and their optimization. Optimization, 61(4), 443–457.

    Google Scholar 

  • Weber, G. W., Batmaz, İ, Köksal, G., Taylan, P., & Yerlikaya-Özkurt, F. (2012b). CMARS: A new contribution to nonparametric regression with multivariate adaptive regression splines supported by continuous optimization. Inverse Problems in Science and Engineering, 20(3), 371–400.

    Google Scholar 

  • Wei, L., Tian, W., Silva, E. A., Choudhary, R., Meng, & Q.-X., & Yang, S. (2015). Comparative study on machine learning for urban building energy analysis. Procedia Engineering, 121, 285–292.

    Google Scholar 

  • Wheeler, D. (2009). Simultaneous coefficient penalization and model selection in geographically weighted regression: The geographically weighted LASSO. Environment and Planning A, 39(3), 722–742.

    Google Scholar 

  • Xiong, R., & Meullenet, J. F. (2004). Application of multivariate adaptive regression splines (MARS) to the preference mapping of cheese sticks. Journal of food science, 69(4).

  • Xu, Q. S., Massart, D. L., Liang, Y. Z., & Fang, K. T. (2003). Two-step multivariate adaptive regression splines for modeling a quantitative relationship between gas chromatography retention indices and molecular descriptors. Journal of Chromatography A, 998(1), 155–167.

    Google Scholar 

  • Yang, C. C., Prasher, S. O., Lacroix, R., & Kim, S. H. (2003). A multivariate adaptive regression splines model for simulation of pesticide transport in soils. Biosystems Engineering, 86(1), 9–15.

    Google Scholar 

  • Yang, D. (2018). Ultra-fast preselection in LASSO-type spatio-temporal solar forecasting problems. Solar Energy, 176, 788–796.

    Google Scholar 

  • Yıldırım, M. H., Özmen, A., Türker Bayrak, Ö, & Weber, G.-W. (2011). Electricity price modeling for Turkey. In Operations research proceedings 2011, selected papers of the international conference on operations tesearch (OR 2011) (pp. 39–44).

  • Yılmaz, Y., Kurz, R., Özmen, A., & Weber, G.-W. (2015). A new algorithm for scheduling condition-based maintenance of gas turbines. In ASME Turbo Expo 2015: Turbine technical conference and exposition volume 9: Oil and gas applications conference proceedings.

  • Zareipour, H., Bhattacharya, K., & Canizares, C. (2006). Forecasting the hourly Ontario energy price by multivariate adaptive regression splines. In Power engineering society general meeting. IEEE.

  • Zhang, Y., Ma, F., & Wang, Y. (2019). Forecasting crude oil prices with a large set of predictors: Can LASSO select powerful predictors? Journal of Empirical Finance, 54, 97–117.

    Google Scholar 

  • Zhou, Y., & Leung, H. (2007). Predicting object-oriented software maintainability using multivariate adaptive regression splines. Journal of Systems and Software, 80(8), 1349–1361.

    Google Scholar 

Download references

Acknowledgements

The author would like to thank the anonymous reviewers for their valuable comments and advice served for an improvement of this study.

Author information

Authors and Affiliations

  1. Institute of Applied Mathematics, METU, Ankara, Turkey

    Ayşe Özmen

Authors
  1. Ayşe Özmen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ayşe Özmen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Özmen, A. Sparse regression modeling for short- and long‐term natural gas demand prediction . Ann Oper Res 322, 921–946 (2023). https://doi.org/10.1007/s10479-021-04089-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-021-04089-x

Keywords

Search

Navigation