Skip to main content
Springer Nature Link
Log in

An Inexact Accelerated Proximal Gradient Method and a Dual Newton-CG Method for the Maximal Entropy Problem

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

This paper describes an algorithm to solve large-scale maximal entropy problems. The algorithm employs an inexact accelerated proximal gradient method to generate an initial iteration point which is important; then it applies the Newton-CG method to the dual problem. Numerical experiments illustrate that the algorithm can supply an acceptable and even highly accurate solution, while algorithms without generating a good initial point may probably fail.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Bebiano, N., Providencia, J.D. Jr., Lemos, R.: Matrix inequalities in statistical mechanics. Linear Algebra Appl. 376, 265–273 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  2. Gray, R.M.: Entropy and Information Theory. Springer, New York (1990)

    Book  MATH  Google Scholar 

  3. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106, 620–630 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  4. Kullback, S.: Information Theory and Statistics. Wiley, New York (1959)

    MATH  Google Scholar 

  5. Christodoulou, S.E.: Traffic modeling and college-bus routing using entropy maximization. J. Transp. Eng. 136, 102–109 (2010)

    Article  Google Scholar 

  6. German, A., Jenkin, M.R., Lespérance, Y.: In: The 2nd Canadian Conference on Entropy-based Image Merging, Computer and Robot Vision, Proceedings (2005)

    Google Scholar 

  7. Hong, H., Schonfeld, D.: Maximum-entropy expectation-maximization algorithm for image reconstruction and sensor field estimation. IEEE Trans. Image Process. 17, 897–907 (2008)

    Article  MathSciNet  Google Scholar 

  8. Narayan, R., Nityananda, R.: Maximum entropy image restoration in astronomy. Annu. Rev. Astron. Astrophys. 24, 127–170 (1986)

    Article  Google Scholar 

  9. Skilling, J., Bryan, R.K.: Maximum entropy image reconstruction: general algorithm. Mon. Not. R. Astron. Soc. 211, 111–124 (1984)

    MATH  Google Scholar 

  10. Wang, Y.M., Wahl, F.M.: Vector-entropy optimization-based neural-network approach to image reconstruction from projections. IEEE Trans. Neural Netw. 8, 1008–1014 (1997)

    Article  Google Scholar 

  11. Chouzenoux, É., Moussaoui, S., Idier, J., Mariette, F.: Efficient maximum entropy reconstruction of nuclear magnetic resonance T1–T2 spectra. IEEE Trans. Signal Process. 58, 6040–6051 (2010)

    Article  MathSciNet  Google Scholar 

  12. Censor, Y.: Entropy optimization via entropy projections. Lect. Notes Control Inf. 38, 450–454 (1982)

    Article  Google Scholar 

  13. Eriksson, J.: A note on solution of large sparse maximum entropy problems with linear equality constraints. Math. Program. 18, 146–154 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  14. Fang, S.C.: An unconstrained convex programming view of linear programming. Z. Oper. Res. Meth. Model Oper. Res. 36, 149–161 (1992)

    Article  MATH  Google Scholar 

  15. Fang, S.C., Tsao, H.S.J.: Linear programming with entropic perturbation. Z. Oper. Res. Meth. Model Oper. Res. 37, 171–186 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  16. Fang, S.C., Rajasekera, J.R., Tsao, H.S.J.: Entropy Optimization and Mathematical Programming. Springer, Berlin (1997)

    Book  MATH  Google Scholar 

  17. Powell, M.J.D.: An algorithm for maximizing entropy subject to simple bounds. Math. Program. 42, 171–180 (1988)

    Article  MATH  Google Scholar 

  18. Amato, U., Hughes, W.: Maximum entropy regularization for Fredholm integral equations of the first kind. Inverse Probl. 7, 793–808 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  19. Yu, N.: A method for solving the convex programming problem with convergence rate O(1/k 2). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)

    MathSciNet  Google Scholar 

  20. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  21. Yu, N.: Smooth minimization of nonsmooth functions. Math. Program. 103, 127–152 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  22. Yu, N.: Gradient methods for minimizing composite objective functions. CORE Discussion Papers (2007)

  23. Toh, K.-C., Yun, S.W.: An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac. J. Optim. 6, 615–640 (2010)

    MathSciNet  MATH  Google Scholar 

  24. Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization. Submitted to SIAM J. Optim. (2008)

  25. Jiang, K.F., Sun, D.F., Toh, K.-C.: An inexact accelerated proximal gradient method for large scale linearly constrained convex SDP. Available at http://www.optimization-online.org/DB_HTML/2011/09/3156.html

  26. Qi, H.D., Sun, D.F.: A quadratically convergent newton method for computing the nearest correlation matrix. SIAM J. Matrix Anal. Appl. 28, 360–385 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  27. Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Program. 58, 353–368 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  28. Bhatia, R.: Matrix Analysis. Springer, Berlin (1997)

    Book  Google Scholar 

  29. Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1994)

    MATH  Google Scholar 

  30. Chen, X., Qi, H.D., Tseng, P.: Analysis of nonsmooth symmetric-matrix-valued functions with applications to semidefinite complementary problems. SIAM J. Optim. 13, 960–985 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  31. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)

    MATH  Google Scholar 

  32. Meng, F.W., Sun, D.F., Zhao, G.Y.: Semismoothness of solutions to generalized equations and the Moreau–Yosida regularization. Math. Program. 104, 561–581 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  33. Zhao, X.Y., Sun, D.F., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  34. Dennis, J.E. Jr., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM, New York (1983)

    MATH  Google Scholar 

  35. Wang, C.J., Sun, D.F., Toh, K.-C.: Solving log-determinant optimization problems by a Newton-CG primal proximal point algorithm. SIAM J. Optim. 20, 2994–3013 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Chengjing Wang thanks Professors Defeng Sun and Kim-Chuan Toh at the National University of Singapore for valuable discussions on this paper during his stint in Singapore. Many thanks also go to Dr. Saïd Moussaoui at the Ecole Centrale de Nantes for sharing with us his Matlab code and for fruitful discussions. Finally, we thank the two anonymous referees and the Editor-in-chief Professor Franco Giannessi for their helpful comments and suggestions which improved the quality of this paper.

Chengjing Wang’s work was supported by the National Natural Science Foundation of China under grant 11201382, the Youth Fund of Humanities and Social Sciences of the Ministry of Education under grant 12YJC910008 and the Fundamental Research Funds for the Central Universities under grants SWJTU12CX055 and SWJTU12ZT15.

Aimin Xu’s work was supported by the National Natural Science Foundation of China under grant 11201430, the Zhejiang Province Natural Science Foundation under grant Y6110310, the Research Funds of the Education Department of Zhejiang Province under grant Y201016720, and the Ningbo Natural Science Foundation under grant 2012A610036.

Author information

Authors and Affiliations

  1. School of Mathematics and Institute of Applied Mathematics, Southwest Jiaotong University, Chengdu, 610031, China

    Chengjing Wang

  2. Institute of Mathematics, Zhejiang Wanli University, Ningbo, 315100, China

    Aimin Xu

Authors
  1. Chengjing Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Aimin Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Aimin Xu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, C., Xu, A. An Inexact Accelerated Proximal Gradient Method and a Dual Newton-CG Method for the Maximal Entropy Problem. J Optim Theory Appl 157, 436–450 (2013). https://doi.org/10.1007/s10957-012-0150-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-012-0150-2

Keywords