Abstract
It is frequently observed that effective exploitation of problem structure plays a significant role in computational procedures for solving large-scale nonlinear optimization problems. A necessary step in this regard is to express the computation in a manner that exposes the exploitable structure. The formulation of large-scale problems in many scientific applications naturally give rise to “structured” representation. Examples of computationally useful structures arising in large-scale optimization problems include unary functions, partially separable functions, and factorable functions. These structures were developed from 1967 through 1990. In this paper we closely examine commonly occurring structures in optimization with regard to efficient and automatic calculation of first- and higher-order derivatives. Further, we explore the relationship between source code transformation as in algorithmic differentiation (AD) and factorable programming. As an illustration, we consider some classical examples.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Griewank, A., Toint, Ph.L.: On the unconstrained optimization of partially separable functions. In: Powell, M.J.D. (ed.) Nonlinear Optimization 1981, pp. 301–312. Academic Press, New York (1982)
Conn, A.R., Gould, N.I.M., Toint, Ph.L.: An introduction to the structure of large scale nonlinear optimization problems and the LANCELOT project. In: Glowinski, R., Lichnewsky, A. (eds.) Computing Methods in Applied Sciences and Engineering, pp. 42–51. SIAM, Philadelphia (1990)
Conn, A.R., Gould, N.I.M., Toint, Ph.L.: LANCELOT: A Fortran Package for Large-Scale Nonlinear Optimization (Release A), 1st edn. Springer, Berlin (1992)
Conn, A.R., Gould, N.I.M., Toint, Ph.L.: Improving the decomposition of partially separable functions in the context of large-scale optimization: a first approach. In: Hager, W.W., Hearn, D.W., Pardalos, P.M. (eds.) Large Scale Optimization: State of the Art, pp. 82–94. Kluwer Academic Publishers, Amsterdam (1994)
Bouaricha, A., Morè, J.J.: Impact of partial separability on large-scale optimization. Comput. Optim. Appl. 7, 27–40 (1997)
Gay, D.M.: More AD of nonlinear AMPL models: computing Hessian information and exploiting partial separability. In: Berz, M., Bischof, C., Corliss, G., Griewank, A. (eds.) Computational Differentiation: Techniques, Applications, and Tools, pp. 173–184. SIAM, Philadelphia (1996)
Conforti, D., De Luca, L., Grandinetti, L., Musmanno, R.: A parallel implementation of automatic differentiation for partially separable functions using PVM. Parallel Comput. 22, 643–656 (1996)
McCormick, G.P., Sofer, A.: Optimization with unary functions. Math. Program. 52(1), 167–178 (1991)
Steihaug, T., Suleiman, S.: Global convergence and the Powell singular function. J. Glob. Optim. 1–9 (2012). doi:10.1007/s10898-012-9898-z. http://www.dx.doi.org/10.1007/s10898-012-9898-z
Hascoët, L., Hossain, S., Steihaug, T.: Structured computation in optimization and algorithmic differentiation. ACM Commun. Comput. Algebra 46(3) (2012)
Ghaemi, A., McCormick, G.P.: Symbolic factorable SUMT: What is it? How is it used? Technical Report T-402. Institute for Management Science and Engineering, The George Washington University, Washington DC (May 1979)
Kedem, G.: Automatic differentiation of computer programs. ACM Trans. Math. Softw. 6(2), 150–165 (1980)
Jackson, R.H.F., McCormick, G.P.: The polyadic structure of factorable function tensors with application to high-order minimization techniques. J. Optim. Theory Appl. 51(1), 63–94 (1986)
Jackson, R.H.F., McCormick, G.P.: Second-order sensitivity analysis in factorable programming: theory and applications. Math. Program. 41(1–3), 1–27 (1988)
Rall, L.B.: Automatic Differentiation: Techniques and Applications. Lecture Notes in Computer Science, vol. 120. Springer, Berlin (1981)
Smith, E.M., Pantelides, C.C.: Global optimisation of nonconvex minlps. Comput. Chem. Eng. 21(Suppl.), S791–S796 (1997)
Goldfarb, D., Wang, S.Y.: Partial-update Newton methods for unary, factorable, and partially separable optimization. SIAM J. Optim. 3(2), 382–397 (1993)
McCormick, G.P.: Nonlinear Programming: Theory, Algorithms and Applications. Wiley, New York (1983)
McCormick, G.P.: Computability of global solutions to factorable nonconvex programs: part I convex underestimating problems. Math. Program. 10(1), 147–175 (1976)
Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. 2nd edn, Number 105 in Other Titles in Applied Mathematics. SIAM, Philadelphia (2008)
McCormick, G.P.: A mini-manual for use of the SUMT computer program and the factorable programming language. Technical Report SOL 74–15. Department of Operations Research, Stanford University, Stanford (August 1974)
Mylander, W.C., Holmes, R., McCormick, G.P.: A Guide to SUMT-Version 4: The Computer Program Implementing the Sequential Unconstrained Minimization Technique for Nonlinear Programming. RAC-P-63, Research Analysis Corporation, McLean (1971)
Pugh, R.E.: A language for nonlinear programming problems. Math. Program. 2, 176–206 (1972)
Fiacco, A.V., McCormick, G.P.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Wiley, New York (1968)
McCormick, G.P.: Minimizing structured unconstrained functions. Technical Paper RAC-TP-277. Research Analysis Corporation, McLean, Virginia (October 1967)
Hascoët, L., Pascual, V.: Tapenade 2.1 user’s guide. Technical Report 0300, INRIA (2004)
Hascoët, L.: Reversal strategies for adjoint algorithms. In: Bertot, Y., Huet, G., Lévy, J.-J., Plotkin, G. (eds.) From Semantics to Computer Science. Essays in Memory of Gilles Kahn, pp. 487–503. Cambridge University Press, New York (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this paper
Cite this paper
Steihaug, T., Hossain, S., Hascoët, L. (2013). Structure in Optimization: Factorable Programming and Functions. In: Gelenbe, E., Lent, R. (eds) Computer and Information Sciences III. Springer, London. https://doi.org/10.1007/978-1-4471-4594-3_46
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4594-3_46
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4593-6
Online ISBN: 978-1-4471-4594-3
eBook Packages: EngineeringEngineering (R0)