Abstract
In this paper, we propose and analyze an algorithm that couples the gradient method with a general exterior penalization scheme for constrained or hierarchical minimization of convex functions in Hilbert spaces. We prove that a proper but simple choice of the step sizes and penalization parameters guarantees the convergence of the algorithm to solutions for the optimization problem. We also establish robustness and stability results that account for numerical approximation errors, discuss implementation issues and provide examples in finite and infinite dimension.
Similar content being viewed by others
References
Attouch, H., Czarnecki, M.-O.: Asymptotic behavior of coupled dynamical systems with multiscale aspects. J. Differ. Equ. 248(6), 1315–1344 (2010)
Attouch, H., Czarnecki, M.-O., Peypouquet, J.: Prox-penalization and splitting methods for constrained variational problems. SIAM J. Optim. 21(1), 149–173 (2011)
Auslender, A., Crouzeix, J.-P., Fedit, P.: Penalty-proximal methods in convex programming. J. Optim. Theory Appl. 55(1), 1–21 (1987)
Alvarez, F., Cominetti, R.: Primal and dual convergence of a proximal point exponential penalty method for linear programming. Math. Program., Ser. A 93(1), 87–96 (2002)
Cominetti, R., Courdurier, M.: Coupling general penalty schemes for convex programming with the steepest descent method and the proximal point algorithm. SIAM J. Optim. 13, 745–765 (2002)
Kiwiel, K.: Exact penalty functions in proximal bundle methods for constrained convex nondifferentiable minimization. Math. Program., Ser. B 52(2), 285–302 (1991)
Peypouquet, J.: Asymptotic convergence to the optimal value of diagonal proximal iterations in convex minimization. J. Convex Anal. 16(1), 277–286 (2009)
Attouch, H., Czarnecki, M.-O., Peypouquet, J.: Coupling forward-backward with penalty schemes and parallel splitting for constrained variational inequalities. SIAM J. Optim. (in press)
Bertsekas, D.: Nonlinear Programming. Athena Scientific, Belmont (1999)
Rockafellar, R.-T.: Convex Analysis. Princeton University Press, Princeton NJ (1970)
Combettes, P.-L.: Quasi-Fejérian analysis of some optimization algorithms. In: Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently Parallel Algorithms for Feasibility and Optimization, pp. 115–152. Elsevier, Amsterdam (2001)
Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Am. Math. Soc. 73, 591–597 (1967)
Baillon, J.-B., Haddad, G.: Quelques propriétés des opérateurs angle-bornés et n-cycliquement monotones. Isr. J. Math. 26(2), 137–150 (1977)
Alvarez, F., Peypouquet, J.: Asymptotic almost-equivalence and ergodic convergence of Lipschitz evolution systems in Banach spaces. Nonlinear Anal. 73(9), 3018–3033 (2010)
Brézis, H.: Analyse fonctionnelle: théorie et applications. Dunod, Paris (1999)
Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Alternating proximal algorithms for weakly coupled convex minimization problems, Applications to dynamical games and PDEs. J. Convex Anal. 15(3), 485–506 (2008)
Attouch, H., Cabot, A., Frankel, P., Peypouquet, J.: Alternating proximal algorithms for constrained variational inequalities. Application to domain decomposition for PDE’s. Nonlinear Anal., Theory, Methods Appl. 74(18), 7455–7473 (2011)
Xu, M.-H.: Proximal alternating directions method for structured variational inequalities. J. Optim. Theory Appl. 134, 107–117 (2007)
Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program., Ser. A 64(1), 81–101 (1994)
Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. Classics in Applied Mathematics, vol. 28. SIAM, Philadelphia (1999)
Evans, L.: Partial Differential Equations, 2nd edn. AMS Graduate Studies in Mathematics. AMS, Providence (2002)
Candès, E.-J., Romberg, J.-K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Jean-Pierre Crouzeix.
Rights and permissions
About this article
Cite this article
Peypouquet, J. Coupling the Gradient Method with a General Exterior Penalization Scheme for Convex Minimization. J Optim Theory Appl 153, 123–138 (2012). https://doi.org/10.1007/s10957-011-9936-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-011-9936-x