Abstract
Simultaneous perturbation stochastic approximation (SPSA) belongs to the class of gradient-free optimization methods that extract gradient information from successive objective function evaluation. This paper describes an improved SPSA algorithm, which entails fuzzy adaptive gain sequences, gradient smoothing, and a step rejection procedure to enhance convergence and stability. The proposed fuzzy adaptive simultaneous perturbation approximation (FASPA) algorithm is particularly well suited to problems involving a large number of parameters such as those encountered in nonlinear system identification using neural networks (NNs). Accordingly, a multilayer perceptron (MLP) network with popular training algorithms was used to predicate the system response. We found that an MLP trained by FASPSA had the desired accuracy that was comparable to results obtained by traditional system identification algorithms. Simulation results for typical nonlinear systems demonstrate that the proposed NN architecture trained with FASPSA yields improved system identification as measured by reduced time of convergence and a smaller identification error.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. T. Abdulsadda, K. Iqbal. A wavelet-based recurrent fuzzy neural network trained with stochastic optimization algorithm. In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, IEEE, San Antonio, USA, pp. 4089–4093, 2009.
A. Boubakir, B. Fares, S. Labiod. A neuro-fuzzy-sliding mode controller using nonlinear sliding surface applied to the coupled ranks system. International Journal of Automation and Computing, vol. 6, no. 1, pp. 72–80, 2009.
D. Necsulescu, Y. W. Jiang, B. Kim. Neural network based feedback linearization control of an unmanned aerial vehicle. International Journal of Automation and Computing, vol. 4, no. 1, pp. 71–79, 2007.
K. Hornik, S. Maxwell, W. Halaber. Universal approximation of an unknown mapping and its derivatives using multilayer feed-forward networks. Neural Networks, vol. 3, no. 5, pp. 551–560, 1990.
K. S. Narendra, K. Parthasarathy. Identification and control of dynamical systems using neural networks. In Proceedings of IEEE Multi-conference on Systems and Control, IEEE, San Antonio, USA, pp. 1295–3001, 2008.
S. Chen, S. A. Billings. Neural networks for nonlinear dynamic system modelling and identification. International Journal of Control, vol. 56, no. 2, pp. 319–346, 1992.
J. C. Spall. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control, vol. 37, no. 3, pp. 332–341, 1992.
J. C. Spall. Developments in stochastic optimization algorithms with gradient approximation based on function measurement. In Proceedings of Winter Simulation Conference, IEEE, Orlando, USA, pp. 207–214, 1994.
A. Vande, C. Renotte, P. Bogaerts, M. Remy. Application of SPSA techniques in nonlinear system identification, [Online], Available: http://www.jhuapl.edu/spsa/PDF-SPSA/VandeWouwerApplicationSPSA.pdf, April 27, 2011.
S. Samarasinghe. Neural Networks for Applied Science and Engineering, USA: Taylor & Francis Group, 2007.
F. Herrera, M. Lozano. Fuzzy adaptive genetic algorithms: Design, taxonomy, and future directions. Soft Computing, vol. 7, no. 8, pp. 545–562, 2003.
J. F. M. Amaral, M. M. Vellasco, R. Tanscheit, M. A. C. Pacheco. A neuro-fuzzy-genetic system for automatic setting control system. In Proceedings of IFSAWorld Congress and the 20th NAFIPS International Conference, IEEE, Vancouver, Canada, vol. 3, pp. 1553–1558, 2001.
N. Xiong, L. Litz, H. Ressom. Learning premises of fuzzy rules for knowledge acquisition in classifications problems. Knowledge and Information Systems, vol. 4, no. 1, pp. 96–111, 2002.
B. Subudhi, D. Jena. An improved differential evolution trained neural network scheme for non linear system identification. International Journal of Automation and Computing, vol. 6, no. 2, pp. 137–144, 2009.
G. E. P. Box, G. M. Jenkins. Time Series Analysis, Forecasting and Control, San Francisco, USA: Holden Day, 1970.
Author information
Authors and Affiliations
Corresponding author
Additional information
Ahmad T. Abdulsadda received the B. Sc. degree in electrical engineering from Tickrit University, Iraq in 1997 and M. Sc. degree in electrical engineering from Baghdad University, Iraq in 2000. From 2000 to 2006, he was a faculty member at Baghdad University, Iraq. Since 2006, he has been a faculty member at Technical Najaf College, Iraq. Currently, he is a Ph.D. candidate in the Smart Micro-systems Laboratory, Electrical and Computer Engineering Department at Michigan State University, Michigan, USA. He received scholarship award from the Ministry of Higher Education in Iraq to get Ph.D. degree from USA. He is a student member of IEEE.
His research interests include robotics fish, feedback control systems, nonlinear estimation techniques, and control theory.
Kamran Iqbal received the B. Sc. degree in avionics engineering from the NED University, Pakistan, in 1980. He received the M. Sc. and Ph.D. degrees in electrical engineering from the Ohio State University, USA in 1988 and 1992. He has been a faculty member at the University of Arkansas at Little Rock, USA since 2000.
His research interests include control systems, industrial process control, biomechanics, motor control, postural stability, and movement coordination.
Rights and permissions
About this article
Cite this article
Abdulsadda, A.T., Iqbal, K. An improved SPSA algorithm for system identification using fuzzy rules for training neural networks. Int. J. Autom. Comput. 8, 333–339 (2011). https://doi.org/10.1007/s11633-011-0589-x
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-011-0589-x