Penalty variable sample size method for solving optimization problems with equality constraints in a form of mathematical expectation

Abstract

Equality-constrained optimization problems with deterministic objective function and constraints in the form of mathematical expectation are considered. The constraints are approximated by employing the sample average where the sample size varies throughout the iterations in an adaptive manner. The proposed method incorporates variable sample size scheme with cumulative and unbounded sample into the well- known quadratic penalty iterative procedure. Line search is used for globalization and the sample size is updated in a such way to preserve the balance between two types of errors—errors coming from the sample average approximation and the approximation of the optimal point. Moreover, the penalty parameter is also updated in an adaptive way. We prove that the proposed algorithm pushes the sample size and the penalty parameter to infinity which further allows us to prove the almost sure convergence towards a Karush-Kuhn-Tucker optimal point of the original problem under the rather standard assumptions. Numerical comparison on a set of relevant problems shows the advantage of the proposed adaptive scheme over the heuristic (predetermined) sample scheduling in terms of number of function evaluations as a measure of the optimization cost.