Skip to main content
SpringerLink
Log in

A Trust Region SQP Algorithm for Equality Constrained Parameter Estimation with Simple Parameter Bounds

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We describe a new algorithm for a class of parameter estimation problems, which are either unconstrained or have only equality constraints and bounds on parameters. Due to the presence of unobservable variables, parameter estimation problems may have non-unique solutions for these variables. These can also lead to singular or ill-conditioned Hessians and this may be responsible for slow or non-convergence of nonlinear programming (NLP) algorithms used to solve these problems. For this reason, we need an algorithm that leads to strong descent and converges to a stationary point. Our algorithm is based on Successive Quadratic Programming (SQP) and constrains the SQP steps in a trust region for global convergence. We consider the second-order information in three ways: quasi-Newton updates, Gauss-Newton approximation, and exact second derivatives, and we compare their performance. Finally, we provide results of tests of our algorithm on various problems from the CUTE and COPS sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J.S. Albuquerque, “Parameter estimation and data reconciliation for dynamic systems,” PhD thesis, Carnegie Mellon University, 1996.

  2. N. Arora and L.T. Biegler, “Redescending estimators for data reconciliation and parameter estimation,” Computers and Chemical Engineering, vol. 25, p. 1585, 2001.

    Google Scholar 

  3. G. Bader and U Ascher, “A new basis implementation for a mixed order boundary value ODE solver,” SIAM J. Scientific and Statistical Computing, vol. 8, p. 483, 1987.

    Google Scholar 

  4. L.T. Biegler, J. Nocedal, C. Schmid, and D.J. Ternet, “Numerical experience with a reduced hessian method for large scale constrained optimization,” Computational Optimization and Applications, vol. 15, p. 45, 2000.

    Google Scholar 

  5. L.T. Biegler, J. Nocedal, and C. Schmid, “A reduced Hessian method for large-scale constrained optimization,” SIAM J. Optimization, vol. 5, no. 2, p. 314, 1995.

    Google Scholar 

  6. A. Bjorck, E. Grimme, and P. van Dooren, “An implicit shift bidiagonalization algorithm for ill-posed systems,” BIT, vol. 34, p. 510, 1994.

    Google Scholar 

  7. I. Bongartz, A.R. Conn, N.I.M. Gould, and Ph.L. Toint, “CUTE: Constrained and unconstrained testing environment,” ACM Transactions on Mathematical Software, vol. 21, no. 1, p. 123, 1995.

    Google Scholar 

  8. D.S. Bunch, D.M. Gay, and R.E. Welsch, “Algorithm 717: Subroutines for maximum likelihood and quasi-likelohood estimation of parameters in nonlinear regression models,” ACM Transactions on Mathematical Software, vol. 19, no. 1, p. 109, 1993.

    Google Scholar 

  9. R.H. Byrd, M.E. Hribar, and J. Nocedal, “An interior point algorithm for large scale nonlinear programming,” SIAM J. Optimization, vol. 9, no. 4, pp. 877-900, 1999.

    Google Scholar 

  10. R.H. Byrd and J. Nocedal, “An analysis of reduced hessian methods for constrained optimization,” Mathematical Programming, vol. 49, pp. 285-323, 1991.

    Google Scholar 

  11. D. Calvetti, L. Reichel, and Q. Zhang, “Estimation of the L-curve via Laczos bidiagonalization,” BIT, vol. 39, no. 4, p. 603, 1999.

    Google Scholar 

  12. T.F. Coleman and Y. Li, “An interior trust region approach for nonlinear minimization subject to bounds,” SIAM J. Optimization, vol. 6, pp. 418-445, 1996.

    Google Scholar 

  13. A. Conn, N. Gould, and P. Toint, Trust-region methods, MPS-SIAM Series on Optimization, Philadelphia, PA, 2000.

  14. J.E. Dennis, M. El-Alem, and M.C. Maciel, “A global convergence theory for general trust region based algorithms for equality constrained optimization,” SIAM J. Opt., vol. 7, p. 177, 1997.

    Google Scholar 

  15. J.E. Dennis Jr., D.M. Gay, and R.E. Welsch, “An adaptive nonlinear least-squares algorithm,” ACM Transactions on Mathematical Software, vol. 7, no. 3, p. 348, 1981.

    Google Scholar 

  16. J.E. Dennis, M. Heinkenschloss, and L. Vicente, “Trust-region interior-point sqp algorithms for a class of nonlinear programming problems,” SIAM J. Control and Optimization, vol. 36, no. 5, p. 1750, 1998.

    Google Scholar 

  17. E.D. Dolan and J. Moré, “Benchmarking optimization software with COPS,” Technical report, ANL/MCS-246, Argonne National Laboratory, 2001.

  18. E.D. Dolan and J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming Series A, on-line, 2002.

  19. M. El-Alem, “A global convergence theory for the Celis-Dennis-Tapia trust region algorithm for constrained optimization,” SIAM J. Numerical Analysis, vol. 28, p. 266, 1991.

    Google Scholar 

  20. R. Fletcher, N. Gould, S. Leyffer, P. Toint, and A. Wächter, “Global convergence of a trust region SQP filter algorithm for general nonlinear programming,” SIAM J. Optimization, vol. 13, no. 3, pp. 635-665, 2003.

    Google Scholar 

  21. R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Mathematical Programming, vol. 92, no. 2, p. 239, 2002.

    Google Scholar 

  22. R. Fletcher and C. Xu, “Hybrid methods for nonlinear least squares,” IMA J. of Numerical Analysis, vol. 7, p. 371, 1987.

    Google Scholar 

  23. D.M. Gay, “Computing optimal locally constrained steps,” SIAM J. Scientific and Statistical Computing, vol. 2, no. 2, p. 186, 1981.

    Google Scholar 

  24. D.M. Gay, “A trust-region approach to linearly constrained optimization,” Numerical Analysis Proceedings (Dundee, 1983). D.F. Griffiths, (Ed.), Springer-Verlag, vol. 72, 1983.

  25. S.M. Goldfeld, R.E. Quandt, and H.F. Trotter, “Maximization by quadratic hill climbing,” Econometrica, vol. 34, no. 3, p. 541, 1966.

    Google Scholar 

  26. G. Golub and U. von Matt, “Quadratically constrained least squares and quadratic problems,” Numerische Mathematik, vol. 59, p. 561, 1991.

    Google Scholar 

  27. F. Gomes, M.C. Maciel, and J.M. Martinez, “Nonlinear programming algorithms using trust regions and augmented Lagrangians with nonmonotone penalty parameters,” Mathematical Programming, vol. 84, p. 161, 1999.

    Google Scholar 

  28. A. Griewank, D. Juedes, and J. Utke, “ADOL-C: A package for the automatic differentiation of algorithms written in C/C++,” ACM Transactions on Mathematical Software, vol. 22, no. 2, p. 131, 1996.

    Google Scholar 

  29. M. Gullicksson, “Algorithms for nonlinear least-squares with Applications to orthogonal regression,” Technical report, UMINF-178.90, University of Umea, Sweden, 1990.

    Google Scholar 

  30. P.C. Hansen, Rank Deficient and Discrete Ill-Posed Problems, SIAM: Philadelphia, 1999.

    Google Scholar 

  31. M.D. Hebden, “An algorithm for minimization using exact second derivatives,” Technical report, Atomic Energy Research Establishment report T.P. 515, Harwell, England, 1973.

    Google Scholar 

  32. N. Krejic, J.M. Martinez, M. Mello, and E. Pilotta, “Validation of an augmented lagrangian algorithm with a Gauss-Newton Hessian approximation using a set of hard-spheres problems,” Computational Optimization and Applications, vol. 16, p. 247, 2000.

    Google Scholar 

  33. M. Lalee, J. Nocedal, and T. Plantenga, “On the implementation of an algorithm for large-scale equality constrained optimization,” SIAM J. Optimization, vol. 8, no. 3, p. 682, 1998.

    Google Scholar 

  34. S. Leyffer, R. Fletcher, and P. Toint, “On the global convergence of a filter SQP algorithm,” SIAM J. Optimization, vol. 13, no. 1, p. 44, 2002.

    Google Scholar 

  35. C. Lin and J.J. Moré, “Newton's method for large bound-constrained optimization problems,” SIAM Journal on Optimization, vol. 9, no. 4, p. 1100, 1999.

    Google Scholar 

  36. J.J. Moré, The Levenberg-Marquardt Algorithm: Implementation and Theory, volume Numerical Analysis, Dundee 1977. Springer-Verlag: Berlin, 1977.

    Google Scholar 

  37. J. Nocedal and M.L. Overton, “Projected Hessian updating algorithms for nonlinearly constrained optimization,” SIAM J. Numer. Anal., vol. 22, no. 5, p. 821, 1985.

    Google Scholar 

  38. J. Nocedal and S.J. Wright, Numerical Optimization, Springer: New York, 1999.

    Google Scholar 

  39. E.O. Omojokun, “Trust region algorithms for optimization with nonlinear equality and inequality constraints,” PhD thesis, Department of Computer Science, University of Colorado, Boulder, 1989.

    Google Scholar 

  40. M. Rojas, “A large scale trust region approach to the regularization of discrete ill-posed problems,” PhD thesis, Rice University, 1998.

  41. M. Rojas, S.A. Santos, and D.C. Sorensen, “A matrix free algorithm for the large-scale trust region subproblem,” SIAM J. Optimization, vol. 11, no. 3, p. 611, 2000.

    Google Scholar 

  42. M. Rojas and D.C. Sorensen, “A trust region approach to the regularization of large-scale discrete forms of ill-posed problems,” SIAM J. on Scientific Computing, vol. 23, no. 6, pp. 1842-1860, 2002.

    Google Scholar 

  43. I.B.F. Tjoa, “Simultaneous solution and optimization strategies for data analysis,” PhD thesis, Carnegie Mellon University, 1991.

  44. M. Ulbrich and S. Ulbrich, “Nonmonotone trust region methods for nonlinear equality constrained optimization without a penalty function,” Mathematical Programming, Series B, on-line, 2002.

  45. A.Wächter and L.T. Biegler, “Global and local convergence of a reduced space Quasi-Newton barrier algorithm for large-scale nonlinear programming,” Technical report, CAPD Technical Report B-00-06, Carnegie Mellon University, August, 2000.

  46. A. Wächter and L.T. Biegler, “Global and local convergence of line search filter methods for nonlinear programming,” Technical report, CAPD Technical Report B-01-09, Carnegie Mellon University, August, 2001.

Download references

Author information

Authors and Affiliations

  1. Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Nikhil Arora & Lorenz T. Biegler

Authors
  1. Nikhil Arora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lorenz T. Biegler
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Arora, N., Biegler, L.T. A Trust Region SQP Algorithm for Equality Constrained Parameter Estimation with Simple Parameter Bounds. Computational Optimization and Applications 28, 51–86 (2004). https://doi.org/10.1023/B:COAP.0000018879.40214.11

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:COAP.0000018879.40214.11

Search

Navigation