Skip to main content
SpringerLink
Log in

An Optimization Technique for Solving a Class of Ridge Fuzzy Regression Problems

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this paper, a hybrid scheme based on recurrent neural networks for approximate coefficients (parameters) of ridge fuzzy regression model with LR-fuzzy output and crisp inputs is presented. Here a neural network is first constructed based on some concepts of convex optimization and stability theory. The suggested neural network model guarantees to find the approximate parameters of the ridge fuzzy regression problem. The existence and convergence of the trajectories of the neural network are studied. The Lyapunov stability for the neural network is also shown. To assess the ridge fuzzy regression estimator, the mean squared prediction error with three different well known distances are used. In order to depict the performance of the proposed ridge technique in the presence of multicollinear data, a Monte Carlo simulation is presented. To further determine, an example of a situation in which one variable is a perfect linear combination of the other variable is used to test the applicability of the proposed method. In this study, the performance of the model is evaluated by error parameters and visualized in the Taylor diagram. The predictive ability of the model thus obtained is examined by cross- validation to investigate how well the model fits and predicts every observations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Khashei M, Hejazi SR, Bijari M (2008) A new hybrid artificial neural networks and fuzzy regression model for time series forecasting. Fuzzy Sets Syst 159(7):769–786

    Article  MathSciNet  MATH  Google Scholar 

  2. Chaudhuri A, De K (2009) Time series forecasting using hybrid neuro-fuzzy regression model. In: International workshop on rough sets, fuzzy sets, data mining, and granular-soft computing, pp 369–381, Springer

  3. Azadeh A, Saberi M, Seraj O (2010) An integrated fuzzy regression algorithm for energy consumption estimation with non-stationary data: a case study of iran. Energy 35(6):2351–2366

    Article  Google Scholar 

  4. Azadeh A, Saberi M, Gitiforouz A (2011) An integrated simulation-based fuzzy regression-time series algorithm for electricity consumption estimation with non-stationary data. J Chin Inst Eng 34(8):1047–1066

    Article  Google Scholar 

  5. Ciavolino E, Calcagnì A (2016) A generalized maximum entropy (GME) estimation approach to fuzzy regression model. Appl Soft Comput 38:51–63

    Article  Google Scholar 

  6. Zuo H, Zhang G, Pedrycz W, Behbood V, Lu J (2017) Granular fuzzy regression domain adaptation in takagi-sugeno fuzzy models. IEEE Trans Fuzzy Syst 26(2):847–858

    Article  Google Scholar 

  7. Ferraro MB (2017) On the generalization performance of a regression model with imprecise elements. Int J Uncertain Fuzziness Knowl Based Syst 25(05):723–740

    Article  MathSciNet  MATH  Google Scholar 

  8. Asai H, Tanaka S, Uegima K (1982) Linear regression analysis with fuzzy model. IEEE Trans Syst Man Cybern 12(6):903–907

    Article  MATH  Google Scholar 

  9. Gong Y, Yang S, Ma H, Ge M (2018) Fuzzy regression model based on incentre distance and application to employee performance evaluation. Int J Fuzzy Syst 20(8):2632–2639

    Article  Google Scholar 

  10. Rabiei MR, Arashi M, Farrokhi M (2019) Fuzzy ridge regression with fuzzy input and output. Soft Comput 23(23):12189–12198

    Article  MATH  Google Scholar 

  11. Choi SH, Jung HY, Kim H (2019) Ridge fuzzy regression model. Int J Fuzzy Syst 21(7):2077–2090

    Article  MathSciNet  Google Scholar 

  12. Wang T, Shi P, Wang G (2020) Solving fuzzy regression equation and its approximation for random fuzzy variable and their application. Soft Comput 24(2):919–933

    Article  MathSciNet  MATH  Google Scholar 

  13. Chen LH, Nien SH (2020) Mathematical programming approach to formulate intuitionistic fuzzy regression model based on least absolute deviations. Fuzzy Optim Decis Making 19:1–20

    Article  MathSciNet  MATH  Google Scholar 

  14. Arefi M (2020) Quantile fuzzy regression based on fuzzy outputs and fuzzy parameters. Soft Comput 24(1):311–320

    Article  MATH  Google Scholar 

  15. Hoerl AE, Kennard RW (1970) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12(1):55–67

    Article  MATH  Google Scholar 

  16. Lawson CL, Hanson R (1974) Solving least squares problems. Society for Industrial and Applied Mathematics, philadelphia

  17. Frank LE, Friedman JH (1993) A statistical view of some chemometrics regression tools. Technometrics 35(2):109–135

    Article  MATH  Google Scholar 

  18. Melo SP, Kibria B (2020) On some test statistics for testing the regression coefficients in presence of multicollinearity: a simulation study. Stats 3(1):40–55

    Article  Google Scholar 

  19. Lukman AF, Adewuyi E, Månsson K, Kibria BG (2021) A new estimator for the multicollinear poisson regression model: simulation and application. Sci Rep 11(1):1–11

    Article  Google Scholar 

  20. Månsson K, Shukur G, Sjölander P (2014) A new asymmetric interaction ridge (air) regression method. Commun Stat Theory Methods 43(3):616–643

    Article  MathSciNet  MATH  Google Scholar 

  21. Roozbeh M, Arashi M (2016) Shrinkage ridge regression in partial linear models. Commun Stat Theory Methods 45(20):6022–6044

    Article  MathSciNet  MATH  Google Scholar 

  22. Zhang T, Deng Z, Choi KS, Liu J, Wang S (2017) Robust extreme learning fuzzy systems using ridge regression for small and noisy datasets. In: 2017 IEEE international conference on fuzzy systems (FUZZ-IEEE), pp 1–7, IEEE

  23. Firinguetti L, Kibria G, Araya R (2017) Study of partial least squares and ridge regression methods. Commun Stat Simul Comput 46(8):6631–6644

    Article  MathSciNet  MATH  Google Scholar 

  24. Arashi M, Kibria BG, Valizadeh T (2017) On ridge parameter estimators under stochastic subspace hypothesis. J Stat Comput Simul 87(5):966–983

    Article  MathSciNet  MATH  Google Scholar 

  25. Saleh AME, Kibria BG, Geroge F (2019) Comparative study of lasso, ridge regression, preliminary test and stein-type estimators for the sparse gaussian regression model. Stat Optim Inform Comput 7(4):626–641

    MathSciNet  Google Scholar 

  26. Lukman AF, Ayinde K, Kibria BG, Adewuyi ET (2020) Modified ridge-type estimator for the gamma regression model. Commun Stat Simul Comput, pp. 1–15

  27. Suhail M, Chand S, Kibria BG (2020) Quantile based estimation of biasing parameters in ridge regression model. Commun Stat Simul Comput 49(10):2732–2744

    Article  MathSciNet  Google Scholar 

  28. Lukman AF, Ayinde K, Kibria G, Jegede SL (2020) Two-parameter modified ridge-type m-estimator for linear regression model. Sci World J 2020:1–24

  29. Kibria B, Lukman AF (2020) A new ridge-type estimator for the linear regression model: Simulations and applications. Scientifica 2020:1–15

  30. Qasim M, Månsson K, Golam Kibria B (2021) On some beta ridge regression estimators: method, simulation and application. J Stat Comput Simul, 91:1699–1712

  31. Nazemi A (2013) Solving general convex nonlinear optimization problems by an efficient neurodynamic model. Eng Appl Artif Intell 26(2):685–696

    Article  Google Scholar 

  32. Chen JS, Ko CH, Pan S (2010) A neural network based on the generalized fischer-burmeister function for nonlinear complementarity problems. Inf Sci 180(5):697–711

    Article  MathSciNet  MATH  Google Scholar 

  33. Dang C, Leung Y, Gao XB, Chen KZ (2004) Neural networks for nonlinear and mixed complementarity problems and their applications. Neural Netw 17(2):271–283

    Article  MATH  Google Scholar 

  34. Effati S, Ghomashi A, Nazemi A (2007) Application of projection neural network in solving convex programming problems. Appl Math Comput 188(2):1103–1114

    MathSciNet  MATH  Google Scholar 

  35. Effati S, Nazemi A (2006) Neural network models and its application for solving linear and quadratic programming problems. Appl Math Comput 172(1):305–331

    MathSciNet  MATH  Google Scholar 

  36. Han Q, Liao LZ, Qi H, Qi L (2001) Stability analysis of gradient-based neural networks for optimization problems. J Global Optim 19(4):363–381

    Article  MathSciNet  MATH  Google Scholar 

  37. Hu X, Wang J (2006) Solving pseudomonotone variational inequalities and pseudoconvex optimization problems using the projection neural network. IEEE Trans Neural Netw 17(6):1487–1499

    Article  Google Scholar 

  38. Hu X, Wang J (2007) “A recurrent neural network for solving a class of general variational inequalities” and Cybernetics. IEEE Transactions on Systems, Man, Part B (Cybernetics) 37(3):528–539

  39. Kennedy MP, Chua LO (1988) Neural networks for nonlinear programming. IEEE Trans Circuits Syst 35(5):554–562

    Article  MathSciNet  Google Scholar 

  40. Liao LZ, Qi H, Qi L (2001) Solving nonlinear complementarity problems with neural networks: a reformulation method approach. J Comput Appl Math 131(1–2):343–359

    Article  MathSciNet  MATH  Google Scholar 

  41. Nazemi A (2012) A dynamic system model for solving convex nonlinear optimization problems. Commun Nonlinear Sci Numer Simul 17(4):1696–1705

    Article  MathSciNet  MATH  Google Scholar 

  42. Nazemi A, Effati S (2013) An application of a merit function for solving convex programming problems. Comput Ind Eng 66(2):212–221

    Article  Google Scholar 

  43. Nazemi A, Nazemi M (2014) A gradient-based neural network method for solving strictly convex quadratic programming problems. Cogn Comput 6(3):484–495

    Article  Google Scholar 

  44. Xia Y, Leung H, Wang J (2002) A projection neural network and its application to constrained optimization problems. IEEE Trans Circuits Syst I Fundam Theory Appl 49(4):447–458

    Article  MathSciNet  MATH  Google Scholar 

  45. Xia Y, Wang J (2004) A general projection neural network for solving monotone variational inequalities and related optimization problems. IEEE Trans Neural Netw 15(2):318–328

    Article  Google Scholar 

  46. Xia Y, Wang J (2005) A recurrent neural network for solving nonlinear convex programs subject to linear constraints. IEEE Trans Neural Netw 16(2):379–386

    Article  Google Scholar 

  47. Yashtini M, Malek A (2007) Solving complementarity and variational inequalities problems using neural networks. Appl Math Comput 190(1):216–230

    MathSciNet  MATH  Google Scholar 

  48. Zak SH, Upatising V, Hui S (1995) Solving linear programming problems with neural networks: a comparative study. IEEE Trans Neural Netw 6(1):94–104

    Article  Google Scholar 

  49. Chang PT (1997) Fuzzy seasonality forecasting. Fuzzy Sets Syst 90(1):1–10

    Article  Google Scholar 

  50. Tseng FM, Tzeng GH et al (2002) A fuzzy seasonal arima model for forecasting. Fuzzy Sets Syst 126(3):367–376

    Article  MathSciNet  MATH  Google Scholar 

  51. Modarres M, Nasrabadi E, Nasrabadi MM (2005) Fuzzy linear regression models with least square errors. Appl Math Comput 163(2):977–989

    MathSciNet  MATH  Google Scholar 

  52. Abbasbandy S, Otadi M (2006) Numerical solution of fuzzy polynomials by fuzzy neural network. Appl Math Comput 181(2):1084–1089

    MathSciNet  MATH  Google Scholar 

  53. Abbasbandy S, Otadi M, Mosleh M (2008) Numerical solution of a system of fuzzy polynomials by fuzzy neural network. Inf Sci 178(8):1948–1960

    Article  MATH  Google Scholar 

  54. Nasrabadi E, Hashemi SM (2008) Robust fuzzy regression analysis using neural networks. Int J Uncertain Fuzziness Knowl Based Syst 16(04):579–598

    Article  MATH  Google Scholar 

  55. Liu HT, Wang J, He YL, Ashfaq RAR (2017) Extreme learning machine with fuzzy input and fuzzy output for fuzzy regression. Neural Comput Appl 28(11):3465–3476

    Article  Google Scholar 

  56. Yeylaghi S, Otadi M, Imankhan N (2017) A new fuzzy regression model based on interval-valued fuzzy neural network and its applications to management. Beni-Suef Univ J Basic Appl Sci 6(2):106–111

    Google Scholar 

  57. Chukhrova N, Johannssen A (2019) Fuzzy regression analysis: systematic review and bibliography. Appl Soft Comput 84:105708

    Article  Google Scholar 

  58. Mosleh M, Otadi M, Abbasbandy S (2010) Evaluation of fuzzy regression models by fuzzy neural network. J Comput Appl Math 234(3):825–834

    Article  MathSciNet  MATH  Google Scholar 

  59. Karbasi D, Nazemi A, Rabiei M (2020) A parametric recurrent neural network scheme for solving a class of fuzzy regression models with some real-world applications. Soft Comput 24:11159–11187

    Article  Google Scholar 

  60. Fazlollahtabar H, Gholizadeh H (2020) Fuzzy possibility regression integrated with fuzzy adaptive neural network for predicting and optimizing electrical discharge machining parameters. Comput Ind Eng 140:106225

    Article  Google Scholar 

  61. Ishibuchi H, Kwon K, Tanaka H (1995) A learning algorithm of fuzzy neural networks with triangular fuzzy weights. Fuzzy Sets Syst 71(3):277–293

    Article  Google Scholar 

  62. Hayashi Y, Buckley JJ, Czogala E (1993) Fuzzy neural network with fuzzy signals and weights. Int J Intell Syst 8(4):527–537

    Article  MATH  Google Scholar 

  63. Buckley J, Eslami E (1997) Neural net solutions to fuzzy problems: The quadratic equation. Fuzzy Sets Syst 86(3):289–298

    Article  MATH  Google Scholar 

  64. Modarres M, Nasrabadi E, Nasrabadi MM (2004) Fuzzy linear regression analysis from the point of view risk. Int J Uncertain Fuzziness Knowl Based Syst 12(5):635–649

    Article  MathSciNet  MATH  Google Scholar 

  65. Coppi R, DUrso P, Giordani P, Santoro A (2006) Least squares estimation of a linear regression model with lr fuzzy response. Comput Stat Data Anal 51(1):267–286

    Article  MathSciNet  MATH  Google Scholar 

  66. DUrso P, Santoro A (2006) Fuzzy clusterwise linear regression analysis with symmetrical fuzzy output variable. Comput Stat Data Anal 51(1):287–313

    Article  MathSciNet  MATH  Google Scholar 

  67. DUrso P, Massari R, Santoro A (2011) Robust fuzzy regression analysis. Inf Sci 181(19):4154–4174

    Article  MathSciNet  MATH  Google Scholar 

  68. Mosleh M, Otadi M, Abbasbandy S (2011) Fuzzy polynomial regression with fuzzy neural networks. Appl Math Model 35(11):5400–5412

    Article  MathSciNet  MATH  Google Scholar 

  69. Mosleh M, Allahviranloo T, Otadi M (2012) Evaluation of fully fuzzy regression models by fuzzy neural network. Neural Comput Appl 21(1):105–112

    Article  Google Scholar 

  70. Otadi M (2014) Fully fuzzy polynomial regression with fuzzy neural networks. Neurocomputing 142:486–493

    Article  Google Scholar 

  71. Roh SB, Ahn TC, Pedrycz W (2012) Fuzzy linear regression based on polynomial neural networks. Expert Syst Appl 39(10):8909–8928

    Article  Google Scholar 

  72. He YL, Wang XZ, Huang JZ (2016) Fuzzy nonlinear regression analysis using a random weight network. Inf Sci 364:222–240

    Article  MATH  Google Scholar 

  73. He YL, Wei CH, Long H, Ashfaq RAR, Huang JZ (2018) Random weight network-based fuzzy nonlinear regression for trapezoidal fuzzy number data. Appl Soft Comput 70:959–979

    Article  Google Scholar 

  74. Pehlivan NY, Apaydın A (2016) Fuzzy radial basis function network for fuzzy regression with fuzzy input and fuzzy output. Complex Intell Syst 2(1):61–73

    Article  Google Scholar 

  75. Zadeh LA (1965) Fuzzy sets. Inf Control 8(3):338–353

    Article  MATH  Google Scholar 

  76. Snee RD, Marquardt DW (1984) Comment: collinearity diagnostics depend on the domain of prediction, the model, and the data. Am Stat 38(2):83–87

    Google Scholar 

  77. Marquaridt DW (1970) Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation. Technometrics 12(3):591–612

    Article  Google Scholar 

  78. Belsley DA, Kuh E, Welsch RE (2005) Regression diagnostics: Identifying influential data and sources of collinearity, vol 571. John Wiley & Sons, New Jersey

    MATH  Google Scholar 

  79. Bazaraa MS, Sherali HD, Shetty CM (2013) Nonlinear programming: theory and algorithms. John Wiley & Sons, New Jersey

    MATH  Google Scholar 

  80. Nazemi A (2018) A capable neural network framework for solving degenerate quadratic optimization problems with an application in image fusion. Neural Process Lett 47(1):167–192

    Article  Google Scholar 

  81. Xu R (1991) A linear regression model in fuzzy environment. Adv Model Simul 27:31–40

    MATH  Google Scholar 

  82. Diamond P (1988) Fuzzy least squares. Inf Sci 46(3):141–157

    Article  MathSciNet  MATH  Google Scholar 

  83. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, Berlin

    Book  MATH  Google Scholar 

  84. Lu J, Wang R (2009) An enhanced fuzzy linear regression model with more flexible spreads. Fuzzy Sets Syst 160(17):2505–2523

    Article  MathSciNet  MATH  Google Scholar 

  85. Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res Atmos 106(D7):7183–7192

    Article  Google Scholar 

  86. Hesamian G, Akbari MG (2019) Fuzzy lasso regression model with exact explanatory variables and fuzzy responses. Int J Approx Reason 115:290–300

    Article  MathSciNet  MATH  Google Scholar 

  87. Hong DH, Hwang C, Ahn C (2004) Ridge estimation for regression models with crisp inputs and gaussian fuzzy output. Fuzzy Sets Syst 142(2):307–319

    Article  MathSciNet  MATH  Google Scholar 

  88. McDonald GC, Galarneau DI (1975) A monte carlo evaluation of some ridge-type estimators. J Am Stat Assoc 70(350):407–416

    Article  MATH  Google Scholar 

Download references

Funding

This study was not funded by any grant.

Author information

Authors and Affiliations

  1. Faculty of Mathematical Sciences, Shahrood University of Technology, Shahrood, Iran

    Delara Karbasi, Alireza Nazemi & Mohammad Reza Rabiei

Authors
  1. Delara Karbasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alireza Nazemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Reza Rabiei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alireza Nazemi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karbasi, D., Nazemi, A. & Rabiei, M.R. An Optimization Technique for Solving a Class of Ridge Fuzzy Regression Problems. Neural Process Lett 53, 3307–3338 (2021). https://doi.org/10.1007/s11063-021-10538-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10538-2

Keywords

Search

Navigation