Abstract
This paper introduces a new fast, effective and practical model structure construction algorithm for a mixture of experts network system utilising only process data. The algorithm is based on a novel forward constrained regression procedure. Given a full set of the experts as potential model bases, the structure construction algorithm, formed on the forward constrained regression procedure, selects the most significant model base one by one so as to minimise the overall system approximation error at each iteration, while the gate parameters in the mixture of experts network system are accordingly adjusted so as to satisfy the convex constraints required in the derivation of the forward constrained regression procedure. The procedure continues until a proper system model is constructed that utilises some or all of the experts. A pruning algorithm of the consequent mixture of experts network system is also derived to generate an overall parsimonious construction algorithm. Numerical examples are provided to demonstrate the effectiveness of the new algorithms. The mixture of experts network framework can be applied to a wide variety of applications ranging from multiple model controller synthesis to multi-sensor data fusion.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
R.A. Jacobs, M.I. Jordan, S.J. Nowlan, and G.E. Hinton, “Adaptive mixtures of local experts,” Neural Computation, vol. 3, pp. 79–87, 1991.
M.I. Jordan and R.A. Jacobs, “Adaptive mixtures of local experts,” Neural Computation, vol. 6, pp. 181–214, 1994.
S. McGinnity and G.W. Irwin, “Nonlinear state estimation using fuzzy local linear models,” Int. J. Systems Science, vol. 28, no. 7, pp. 643–656, 1996.
Q. Gan and C.J. Harris, “Fuzzy local linearisation and local basis function expansion in nonlinear systems modelling,” IEEE Trans. SMC(B), vol. 129, pp. 559–565, 1999.
L. Breiman, “Stacked regression,” Machine Learning, vol. 24, pp. 49–64, 1996.
M. Meila and M.I. Jordan, “Markov mixture of experts,” in Multiple Model Approaches to Modelling and Control, edited by R. Murray Smith and T.A. Johansen, Taylor and Francis: London, pp. 145–166, 1997.
S. Chen, S.A. Billings, and W. Luo, “Orthorgonal least squares methods and their applications to non-linear system identification,” Int. J. Control, vol. 50, pp. 1873–1896, 1989.
X. Hong and S.A. Billings, “Parameter estimation based on stacked regression and evolutionary algorithms,” IEE Proc D, Control Theory and Applications, vol. 146, no. 5, pp. 406–414, 1999.
H.J. Newton, Timeslab: A Time Series Analysis Laboratory, Wadsworth and Brooks/Cole, USA, 1988.
S. Chen, S.A. Billings, C.F.N. Cowan, and P.M. Grant, “Nonlinear system identification using radial basis functions,” Int. J. Systems Science, vol. 21, no. 12, pp. 2513–2539, 1990.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hong, X., Harris, C. A Mixture of Experts Network Structure Construction Algorithm for Modelling and Control. Applied Intelligence 16, 59–69 (2002). https://doi.org/10.1023/A:1012869427428
Issue Date:
DOI: https://doi.org/10.1023/A:1012869427428