Abstract
The aim of this paper is to present an alternative solution model to estimate the coefficients of large-scaled linear and nonlinear real-life problems due to the fact that least squares and least median squares parameter estimators have some drawbacks when including so many input variables or increased size of the real-world problems. The study presents a hierarchical soft computing model (SOFTC) that consists of three stages. The first stage constitutes a real-valued breeder genetic algorithm (RVBGA). The second stage is constructing a simulated annealing (SA) algorithm in which the best parameter estimation of the RVBGA is selected as its initial point. The third stage is developing a hierarchical soft computing model by using fuzzy recombination method. SOFTC optimizes the best parameter estimations of this algorithms and it provides a trust region for parameter estimation. Three test problems, one of which is linear and others are nonlinear, are used to examine robustness of proposed models. SOFTC, RVBGA_SA and RVBGA algorithms performed the best parameter estimations, respectively, for the three test problems. The results which are discussed in detail are promising for future usage of these algorithms.
Graphical abstract
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abu Argub O (2017) Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm–Volterra integrodifferential equations. Neural Comput Appl 28:1591–1610
Abu Argub O, Abo-Hammour Z (2014) Numerical solution of systems of second-order boundary value problems using continuous genetic algorithm. Inf Sci 279:396–415
Abu Argub O, Al-Smadi HM, Momani S, Hayat T (2016) Numerical solutions of fuzzy differential equations using reproducing kernel Hilbert space method. Soft Comput 20:3283–3302
Abu Argub O, Al-Smadi HM, Momani S, Hayat T (2017) Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems. Soft Comput 21:7191–7206
Akyol AP (2006) Parameter estimation of nonlinear econometric models with genetic algorithm approach. Master’s thesis, Gazi University Institute of Social Sciences, Department of Econometrics, Ankara
Alba Torres E, Khuri S (2004) Applying evolutionary algorithms to combinatorial optimization problems. In: ICCS 2001, LNCS, vol 2074. Springer, Berlin, pp 689–698
Alp A, Çerçioğlu H, Tokaylı MA, Dengiz B (2001) Stokastik montaj hattı dengeleme: Bir tavlama benzetimi algoritması. Endüstri Mühendisliği Dergisi 12 (3–4):32–51
Aydogdu O (2006) Sensorless control of brushless direct current engines with genetic based fuzzy controller. Unpublished PhD thesis, Selcuk University, Graduate School of Natural and Applied Sciences, Department of Electrical and Electronics Engineering, Konya
Back T, Schwefel HP (1993) An overview of evolutionary algorithms for parameter optimization. Evol Comput 1:1–23
Belanche LA (1999) An application example of the breeder genetic algorithm to function optimization. Secció d’Intel-ligencia Artificial, Dept. de Llenguatges i Sistemes Informatics, Universitat Politecnica de Catalunya, Barcelona
Blum C, Roli A (2003) Metaheuristics in combinatorial optimization: overview and conceptual comparison. ACM Comput Surv 35:268–308
Cetisli B, Edizkan R (2011) Estimation of adaptive neuro-fuzzy inference system parameters with the expectation maximization algorithm and extended kalman smoother. Neural Comput Appl 20(3):403–415
Cetisli B, Kalkan H (2011) Polynomial curve fitting with varying real powers. J Electron Electr Eng 6(112):117–122
Ceylan H, Ozturk HK (2004) Estimating energy demand of turkey based on economic indicators using genetic algorithm approach. Energy Convers Manag 45:2525–2537
Daniel C (1999) Fitting equations to data: computer analysis of multifactor data. Wiley, New York
Davidor Y (1991) Genetic algorithms and robotics. World Scientific, Singapore
Davis L (1987) Genetic algorithms and simulated annealing. Pitman, London
Deb K, Jain H (2011) Parent to mean-centric self-adaptation in single and multi-objective real-parameter genetic algorithms with SBX operator. KanGAL Report No. 2011017
Deconinck E, Zhang MH, Petitet F, Dubus E, Ijjaali I, Coomans D et al (2008) Boosted regression trees, multivariate adaptive regression splines and their two-step combinations with multiple linear regression or partial least squares to predict blood-brain barrier passage: a case study. Anal Chim Acta 609(1):13–23
Denison DGT, Mallick BK, Smith FM (1998) Bayesian MARS. Stat Comput 8(4):337–346
Dogan E (2015) Revisiting the relationship between natural gas consumption and economic growth in Turkey. Energy Sources Part B Econ Plan Policy 10(4):361–370
Folly KA, Sheetekela SP (2012) Optimal design of power system controller using breeder genetic algorithm. In: Gao S (ed) Chapter-15, bio-inspired computational algorithms and their applications. Open Access Book. ISBN: 978- 953-51-0214-4
Forouzanfar M, Doustmohammadi A, Menhaj BM, Hasanzadeh S (2010) Modeling and estimation of the natural gas consumption for residential and commercial sectors. Appl Energy 87:268–274
Friedman J (1988) Multivariate adaptive regression splines. Technical report 102, Deparment of Statistics, Stanford University, California (revised 1990)
Galvez A, Iglesias A (2013) A new iterative mutually coupled hybrid GA–PSO approach for curve fitting in manufacturing. Appl Soft Comput 13(3):1491–1504
Ghasemi JB, Zolfonoun E (2013) Application of principal component analysis–multivariate adaptive regression splines for the simultaneous spectrofluorimetric determination of dialkyltins inmicellar media. Spectrochim Acta Part A Mol Biomol Spectrosc 115:357–363
Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading
Gulsen M, Smith AE (1999) A hierarchical genetic algorithm for system identification and curve fitting with a supercomputer implementation. IMA volumes in mathematics and its applications, evolutionary algorithms. Springer, New York, pp 111–138
Gulsen M, Smith AE, Tate DM (1995) A genetic algorithm approach to curve fitting. Int J Prod Res 33(7):1911–1923
Gurunlu Alma O, Vupa O (2008) The comparison o least squares and least median squares estimation methods which are used in linear regression analysis, Suleyman Demirel University. J Sci 3(2):219–229
Haupt RL, Haupt SE (2004) Practical genetic algorithms. Wiley, Hoboken
Herrera F, Lozano M (2001) Adaptive genetic operators based on coevolution with fuzzy behaviors. IEEE Trans Evol Comput 5:2
Herrera F, Lozano M, Verdegay JL (1998) tackling real-coded genetic algorithms: operators and tools for behavioural analysis. Artif Intell Rev 12:265–319
Holland JH (1975) Adaptation in natural and artificial systems. The University of Michigan Press, Ann Arbor
Im CH, Jung HK, Kim YJ (2003) Hybrid genetic algorithm for electromagnetic topology optimization. IEEE Trans Magn 39:2163–2169
Internet (2016) Simulated annealing tutorial. http://apmonitor.com/me575/index.php/Main/SimulatedAnnealing. Accessed 20 Aug 2016
Jang JSR, Sun CT, Mizutani E (1997) Neuro-fuzzy and soft computing. Prentice Hall, Upper Saddle River
Karadede Y (2014) A hybrid algorithm approach to curve fitting problems, graduate school of applied and natural sciences. M.Sc. thesis, Department of Industrial Engineering, Suleyman Demirel University, Isparta, Turkey
Karadede Y, Ozdemir G, Aydemir E (2017) Breeder hybrid algorithm approach for natural gas demand forecasting model. Energy 141:1269–1284
Karr CL, Stanley DA, Scheiner BJ (1991) Genetic algorithm applied to least squares curve fitting. U.S. Bureau of Mines Report of Investigations 9339, Washington
Karr CL, Weck B, Massart DL, Vankeerberghen P (1995) Least median squares curve fitting using a genetic algorithm. Eng Appl Artif Intell 8(2):177–189
Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Sci New Ser 220(4598):671–680
Kriner M (2007) Survival analysis with multivariate adaptive regression splines. Dissertation, LMU Munchen: Faculty of Mathematics, Computer Science and Statistics, Munchen
Lee Y, Wu H (2012) MARS approach for global sensitivity analysis of differential equation models with applications to dynamics of influenza infection. Bull Math Biol 74:73–90
Li J, Ding R (2013) Parameter estimation methods for nonlinear systems. Appl Math Comput 219:4278–4287
Lin CJ, Chen HF, Lee TS (2011) Forecasting tourism demand using time series, artificial neural networks and multivariate adaptive regression splines: evidence from Taiwan. Int J Bus Adm 2(2):14–24
Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21:1087–1092
Mühlenbein H, Schlierkamp-Voosen D (1993) Predictive models for the breeder genetic algorithm: I. Continuous parameter optimization. Evol Comput 1:25–49
Mukhopadhyay SK, Singh MK, Srivastava R (1998) FMS machine loading: a simulated annealing approach. Int J Prod Res 36(6):1529–1547
Ozdemir G, Aydemir E, Olgun MO, Mulbay Z (2016) Forecasting of turkey natural gas demand using a hybrid algorithm. Energy Sources Part B Econ Plan Policy 11(4):295–302
Rogers D (1991) G/SPLLINES: a hybrid of Friedman’s multivariate adaptive regression splines (MARS) algorithms with Holland’s genetic algorithm. In: Proceedings of the fourth international conference on genetic algorithms, pp 384–391
Rousseeuw PJ (1984) least median of squares regression. J Am Stat Assoc 79:871–880
Rousseeuw P, Leroy A (1987) Robust regression and outlier detection. Wiley, Hoboken, pp 84–143
Ryan TP (1997) Modern regression methods. Wiley, New York
Senturk S (2009) Applıed genetic algorithms approach to curve fitting problems. Master’s thesis, The Graduate School of Natural and Applıed Scıences, Computer Engıneerıng, Bahçeşehir University, Istanbul
Sheetekela SP, Folly KA (2010) Power system controller design: a comparison between breeder genetic algorithm and population based incremental learning. In: The 2010 international joint conference on neural networks, neural networks (IJCNN), pp 1–8
Shi X, Shen J, Yoon D (2015) Genetic search for optimally-constrained multiple-line fitting of discrete data points. Appl Soft Comput 40:236–251
Siniksaran E, Aktutun A (2005) Calculating the least median squares line with random strips. Istanbul University, Faculty of Economics. J Econom Stat 1:11–20
Taylan P, Weber G-W, Yerlikaya-Özkurt F (2010) A new approach to multivariate adaptive regression spline by using Tikhonov regularization and continuous optimization. TOP (Oper Res J SEIO (Span Stat Oper Res Soc) 18(2):377–395
Toksari MD (2007) Ant colony optimization approach to estimate energy demand of Turkey. Energy Policy 35:3984–3990
Toksari MD (2010) Predicting the natural gas demand based on economic indicators: case of Turkey. Energy Sources Part A Recovery Util Environ Effects 32:559–566
Unler A (2008) Improvement of energy demand forecasts using swarm intelligence: the case of Turkey with projections to 2025. Energy Policy 36:1937–1944
Unsal MG (2013) Comparison of deterministic simulated annealing methods for quadratic assignment problems. Uludag University. J Fac Eng Archit 18(2):37–46
Weber GW, Batmaz I, Köksal G, Taylan P, Yerlikaya-Özkurt F (2012) CMARS: a new contribution to nonparametric regression with multivariate adaptive regression splines supported by continuous optimization. Inverse Probl Sci Eng 20(3):371–400
Wei B, Xie N, Hu A (2018) Optimal solution for novel grey polynomial prediction model. Appl Math Model. https://doi.org/10.1016/j.apm.2018.06.035
Yannakakis M (1998) Computational complexity, chapter 2, local search in combinatorial optimization. In: Aartz E, Lenstra J (eds) Series in discrete mathematics and optimization. Wiley, New York
Yao X, Liu Y (1999) Evolutionary programming made fast. IEEE Trans Evol Comput 3:82–102
Yazıcı C, Yerlikaya-Özkurt F, Batmaz İ (2015) A computational approach to nonparametric regression: bootstrapping CMARS method. Mach Learn 101(1–3):211–230
Yerlikaya F (2008) A new contribution to nonlinear robust regression and classification with mars and its applications to data mining for quality control in manufacturing. Master thesis, Graduate School of Applied Mathematics, Department of Scientific Computing, METU, Ankara, Turkey
Yerlikaya-Özkurt F, Askan A, Weber G-W (2016) A hybrid computational method based on convex optimization for outlier problems: application to earthquake ground motion prediction. Informatıca 27(4):893–910
York TP, Eaves LJ, Van Den Oord EJ (2006) Multivariate adaptive regression splines: a powerful method for detecting disease-risk relationship differences among subgroups. Stat Med 25(8):1355–1367
Zakeri IF, Adolph AL, Puyau MR, Vohra FA, Butte NF (2010) Multivariate adaptive regression splines models for the prediction of energy expenditure in children and adolescents. J Appl Physchol 108:128–136
Zhang G-L, Li G-Y, Ma J-W (2006) Real-coded genetic algorithm for constrained optimization problem. In: Proceedings of the fifth international conference on machine learning and cybernetics, Dalian, pp 13–16
Acknowledgements
A part of the study reported in this paper was performed within projects funded by TUBITAK (Scientific and Technological Research Council of Turkey) 2210-C program with Grant Number of 1649B021303459 and by Suleyman Demirel University Scientific Research Projects Unit (BAP) with Grant Number of 3626-YL1-13. Authors would like to thank to the editor and the anonyms referees for their valuable comments and criticisms. Their contributions lead to the improved version of this paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Human and animal rights
This study does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Karadede, Y., Özdemir, G. A hierarchical soft computing model for parameter estimation of curve fitting problems. Soft Comput 22, 6937–6964 (2018). https://doi.org/10.1007/s00500-018-3413-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-018-3413-5