Abstract
This paper explores how qualitative information can be used to improve the performance of global optimization procedures. Specifically, we have constructed a nonlinear parameter estimation reasoner (NPER) for finding parameter values that match an ordinary differential equation (ODE) model to observed data. Qualitative reasoning is used within the NPER, for instance, to intelligently choose starting values for the unknown parameters and to empirically determine when the system appears to be chaotic. This enables odrpack, the nonlinear least-squares solver that lies at the heart of this NPER, to avoid terminating at local extrema in the regression landscape. odrpack is uniquely suited to this task because of its efficiency and stability. The NPER's robustness is demonstrated via a Monte Carlo analysis of simulated examples drawn from across the domain of dynamics, including systems that are nonlinear, chaotic, and noisy. It is shown to locate solutions for noisy, incomplete real-world sensor data from radio-controlled cars used in the University of British Columbia's soccer-playing robot project. The parameter estimation scheme described in this paper is a component of pret, an implemented computer program that uses a variety of artificial intelligence techniques to automate system identification— the process of inferring an internal ODE model from external observations of a system — a routine and difficult problem faced by engineers from various disciplines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H.D.I. Abarbanel, Analysis of Observed Chaotic Data(Springer, Berlin, 1995).
K.J. Astrom and P. Eykhoff, System identification–a survey, Automatica 7 (1971) 123–167.
D.G. Bobrow, ed., Qualitative Reasoning about Physical Systems(MIT Press, Cambridge, MA, 1985).
P.T. Boggs, R.H. Byrd, J.E. Rogers and R.B. Schnabel, User’s reference guide for ODRPACK–software for weighted orthogonal distance regression, Internal Report 89-4103, National Institute of Standards and Technology, Gaithersburg, MD (1991).
P.T. Boggs, R.H. Byrd and R.B. Schnabel, A stable and efficient algorithm for nonlinear orthogonal distance regression, SIAM Journal of Scientific and Statistical Computing 8(6) (1987) 1052–1078.
P.T. Boggs, J.W. Tolle and A.J. Kearsley, A practical algorithm for general large scale nonlinear optimization problems, Technical Report 5704, National Institute of Standards and Technology, Gaithersburg, MD (1996). Submitted to SIAM Journal on Optimization.
E. Bradley, Automatic construction of accurate models of physical systems, in: Proceedings of the 8th International Workshop on Qualitative Reasoning about Physical Systems, Nara, Japan (1994).
E. Bradley and M. Easley, Reasoning about sensor data for automated system identification, in: Proceedings of the 2nd International Symposium on Intelligent Data Analysis (IDA-97)(1997).
E. Bradley, M. Easley and A. Hogan, Qualitative observer theory, in preparation.
E. Bradley and R. Stolle, Automatic construction of accurate models of physical systems, Annals of Mathematics and Artificial Intelligence 17 (1996) 1–28.
A.C. Capelo, L. Ironi and S. Tentoni, The need for qualitative reasoning in automated modeling: A case study, in: Proceedings of the 10th Internattional Workshop on Qualitative Reasoning, eds. Y. Iwasaki and A. Farquhar (Stanford Sierra Camp, CA, 1996) pp. 32–39.
B.W. Char, K.O. Geddes, G.H. Gonnet, B.L. Leong, M.B. Monagan and S.M. Watt, Maple V Language Reference Manual(Springer, New York, 1991).
J.E. Dennis, D.M. Gay and R.E. Welsch, An adaptive nonlinear least-squares algorithm, ACM Transactions on Mathematical Software 7(3) (1981) 348–368.
J.E. Dennis and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations(Prentice-Hall, Englewood Cliffs, NJ, 1983).
A. Farquhar and G. Brajnik, A semi-quantitative physics compiler, in: Proceedings of the 8th International Workshop on Qualitative Reasoning about Physical Systems, Nara, Japan (1994).
R. Fletcher, Practical Methods of Optimization(Wiley, New York, 2nd ed., 1987).
K.D. Forbus, Qualitative process theory, Artificial Intelligence 24 (1984) 85–168.
K.D. Forbus, The qualitative process engine, in: Readings in Qualitative Reasoning About Physical Systems, eds. D.S. Weld and J. de Kleer (Morgan Kaufmann, San Mateo, CA, 1990).
P.E. Gill, W. Murray and M.H. Wright, Practical Optimization(Academic Press, Boston, 1981).
R. Horst, P. Pardalos and N. Thoai, Introduction to Global Optimization, Nonconvex Optimization and its Applications, Vol. 3 (Kluwer Academic, Dordrecht, 1995).
E. Hung, Parameter estimation in chaotic systems, Technical Report AIM 1541, MIT Artificial Intelligence Lab (May 1995).
J.-N. Juang, Applied System Identification(Prentice-Hall, Englewood Cliffs, NJ, 1994).
R.E. Kalman, A new approach to filtering and prediction problems, J. Basic Engrg. 82D (1960) 35–45.
P. Langley, H.A. Simon, G.L. Bradshaw and J.M. Zytkow, eds., Scientific Discovery: Computational Explorations of the Creative Process(MIT Press, Cambridge, MA, 1987).
L. Ljung, ed., System Identification: Theory for the User(Prentice-Hall, Englewood Cliffs, NJ, 1987).
J.J. Moré, D.C. Sorensen, K.E. Hillstrom and B.S. Garbow, in: Sources and Development of Mathematical Software, ed. W.J. Cowell (Prentice-Hall, Englewood Cliffs, NJ, 1984). Chapter “The MINPACK project”.
J. Rees and W. Clinger, The revised report on the algorithmic language Scheme, ACM SIGPLAN Notices 21 (1986) 37.
L.F. Shampine and H.A. Watts, DEPAC–design of a user oriented package of ODE solvers, Technical Report SAND79-2374, Sandia National Laboratories (1979).
D.L. Smith, Introduction to Dynamic System Modeling for Design(Prentice-Hall, Englewood Cliffs, NJ, 1994).
R.M. Stolle and E. Bradley, A customized logic paradigm for reasoning about models, in: Proceedings of the 10th International Workshop on Qualitative Reasoning about Physical Systems, Stanford Sierra Camp, CA (1996).
A. Torn and A. Zilinskas, Global Optimization, Lecture Notes in Computer Science 350 (Springer, Berlin, 1989).
B.C. Williams, A theory of interactions: Unifying qualitative and quantitative algebraic reasoning, Artificial Intelligence 51 (1991).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bradley, E., O'Gallagher, A. & Rogers, J. Global solutions for nonlinear systems using qualitative reasoning. Annals of Mathematics and Artificial Intelligence 23, 211–288 (1998). https://doi.org/10.1023/A:1018972409754
Issue Date:
DOI: https://doi.org/10.1023/A:1018972409754