Abstract
The problem of constructing an adequate and parsimonious neural network topology for modeling non-linear dynamic system is studied and investigated. Neural networks have been shown to perform function approximation and represent dynamic systems. The network structures are usually guessed or selected in accordance with the designer’s prior knowledge. However, the multiplicity of the model parameters makes it troublesome to get an optimum structure. In this paper, an alternative algorithm based on a multi-objective optimization algorithm is proposed. The developed neural network model should fulfil two criteria or objectives namely good predictive accuracy and minimum model structure. The result shows that the proposed algorithm is able to identify simulated examples correctly, and identifies the adequate model for real process data based on a set of solutions called the Pareto optimal set, from which the best network can be selected.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hagan MT, Demuth MT, Beale MH (1996) Neural network design. PWS Publishing, Boston
Billings SA, Jamaluddin H, Chen S (1992) Properties of neural networks with applications to modeling non-linear dynamical systems. Int J Control 55(1):193–224
Chen S, Billings SA, Grant PM (1990) Non-linear system identification using neural networks. Int J Control 51(6):1191–1214
Xua KJ, Zhang J, Wang XF, Teng Q, Tan J (2008) Improvements of nonlinear dynamic modeling of hot-film MAF sensor. Sens Actuators A 147:34–40
Caruana R, Lawrence S, Giles CL (2001) Overfitting in neural networks: backpropagation, conjugate gradient, and early stopping. Adv Neural Inf Process Syst 13:402–408
Bebis G, Georgiopoulos M (1994) Feed-forward neural networks: why network size is so important. IEEE Potentials 13(4):27–31
Sietsma J, Dow RJF (1988) Neural net pruning—why and how. In: Proceedings of the IEEE international conference on neural networks, San Diego, pp 325–333
Ahmad R, Jamaluddin H, Hussain MA (2004) Model structure selection for discrete-time nonlinear systems using genetic algorithm. J Syst Control Eng 218(12):85–98
SK Oh, Pedrycz W (2006) Genetic optimization driven multi layer hybrid fuzzy neural networks. Simul Model Pract Theory 14:597–613
Park KJ, Pedrycz W, Oh SK (2007) A genetic approach to modeling fuzzy systems based on information granulation and successive generation-based evolution method. Simul Model Pract Theory 15:1128–1145
Koza YJR, Rice JP (1991) Genetic generation of both the weights and architecture for a neural network. In: IEEE international joint conference on neural networks, vol 2. IEEE Press, Seattle, pp 397–404
Deb K (2001) Multi-objective optimization using evolutionary algorithms. Wiley, Chichester
Schaffer JD (2011) Some experiments in machine learning using vector evaluated genetic algorithm. Ph.D Thesis, Vanderbilt University, Nashville, TN
Fonseca CM, Fleming PJ (1993) Genetic algorithms for multi-objective optimization: formulation, discussion and generalisation. In: Proceedings of the fifth international conference on genetic algorithms, Morgan Kaufman, San Mateo, pp 416–423
Srinivas N, Deb K (1994) Multi-objective optimization using non-dominated sorting in genetic algorithms. Evol Comput 2(3):221–248
Fieldsend JE, Singh S (2005) Pareto evolutionary neural networks. IEEE Trans Neural Netw 16(2):338–354
Abbass HA (2003) Speeding up backpropagation using multiobjective evolutionary algorithms. Neural Comput 15(11):2705–2726
Sexton RS, Dorsey RE, Sikander NA (2004) Simultaneous optimization of neural network function and architecture algorithm. Decis Support Syst 36:283–296
Gonzalez J, Rojas I, Ortega J, Pomares H, Fernandez J, Diaz A (2003) Multi-objective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation. IEEE Trans Neural Netw 14(1):1478–1495
Palmes S (2005) Robustness, evolvability and optimality in evolutionary neural networks. Biosystems 82(2):168–188
Mandal D, Pal SK, Sah P (2007) Modeling of electrical discharge machining process using back propagation neural network and multi-objective optimization using non-dominating sorting genetic algorithm-II. J Mater Process Technol 186:154–162
Sexton RS, Dorsey RE, Johnson JD (1999) Optimization of neural networks: a comparative analysis of the genetic algorithm and simulated annealing. Eur J Oper Res 114:589–601
Ljung L (1999) System identification, theory for the user, 2nd edn. Prentice-Hall, Englewood Cliffs
Ibnkahla M (2003) Nonlinear system identification using neural networks trained with natural gradient descent. EURASIP J Appl Signal Process 12:1229–1237
Funahashi K (1989) On the approximate realization of continuous mappings by neural networks. Neural Netw 2:183–192
Norgaard MRO, Poulsen NK, Hansen LK (2000) Neural networks for modeling and control of dynamic systems. A practitioner’s handbook. Springer, London
Zitzler E, Deb K, Thiele L (2000) Comparison of multi-objective evolutionary algorithm: empirical results. Evol Comput 8:173–195
Billings SA, Voon WSF (1986) Correlation based model validity tests for non-linear models. Int J Control 44(1):235–244
Box GEP, Jenkins GM, Reinsel GC (1994) Time series analysis forecasting and control. Prentice-Hall Inc, Englewood Cliffs
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Loghmanian, S.M.R., Jamaluddin, H., Ahmad, R. et al. Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm. Neural Comput & Applic 21, 1281–1295 (2012). https://doi.org/10.1007/s00521-011-0560-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-011-0560-3