Hongbo Zhou
ABSTRACT
A class of Maximum A Posteriori(MAP) formulations built on various graph models are of great interests for both theoretical and practical applications. Recent advances in this field have extended the connections between the linear program (LP) relaxation and various tree-reweighted message passing algorithms. At both sides, many algorithms and their optimality certificates are proved, provided no conflict exists between the node marginal maximum and the corresponding edge marginal maximum. However, these conflicts are usually inevitable for general non-trivial Markov random fields (MRFs). Our work is aimed at reducing such conflicts by reparameterizing the original energy distributions in pairwise Markov random field. All node potentials will be decomposed and attached to local edges according to their local graph structures. And thus, only edge marginals are needed in our linear program relaxation, and the node marginals are only used to exchange information among different parts of the graph. We incorporated this consistent graph-structured reparameterization into some latest LP optimality guaranteed proximal solvers, and the resulted algorithms outperform the original ones in convergence rate and also have a better behavior to converge to MAP optimality monotonously even for some highly noisy MRFs.
- Bertsekas, D. P. Nonlinear programming. Athena Scientific, Belmont, MA, 1995.Google Scholar
- Bertsimas, D., & Tsitsikilis, J. Introduction to linear optimization. Athena Scientific, Belmont, MA, 1997. Google ScholarDigital Library
- Boyd, S., & L. Vandenberghe, L. Convex optimization. Cambridge University Press, Cambridge, UK, 2004. Google ScholarDigital Library
- Freeman, W. T., & Weiss Y. On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Trans. Info. Theory, 47:736--744, 2001. Google ScholarDigital Library
- Hazan, T., & Shashua, A. Convergent message-passing algorithms for inference over general graphs with convex free energy. In The 24th Conference on Uncertainty in Artificial Intelligence (UAI), Helsinki, 2008.Google Scholar
- Johnson, J. K., Malioutov D. M., Willsky A. S. Lagrangian relaxation for MAP estimation in graphical models. In Proc. 45th Allerton Conf. Comm. Cont. Comp., Urbana-Champaign, IL, September 2007.Google Scholar
- Kolmogorov, V. Convergent tree-reweighted message-passing for energy minimization. In International Workshop on Artificial Intelligence and Statistics, January 2005.Google Scholar
- Kumar, P., Torr, P. H. S., Zisserman A. Solving markov random fields using second order cone programming. IEEE Conference of Computer Vision and Pattern Recognition, 2006. Google ScholarDigital Library
- Lauritzen, S. L. Graphical Models. Oxford University Press, 1996.Google Scholar
- Ravikumar, P., Agarwal, A., Wainwright, M. J. Message-passing for graph-structured linear programs: proximal projections, convergence and rounding schemes. ICML 2008: 800--807. Google ScholarDigital Library
- Schlesinger, M. I. Syntactic analysis of two-dimensional visual signals in noisy conditions. Kibernetika. v4. 113--130, 1976.Google Scholar
- Schlesinger, M. I. Mathematical tools for image processing. Kiev: Nauk. dumka, 1989.Google Scholar
- Schlesinger, M. I., & Flach, B. Some solvable subclass of structural recognition problems. In Czech Pattern Recognition Workshop, 2000.Google Scholar
- Sontag, D., & Jaakkola T. New outer bounds on the marginal polytope. ICML07.Google Scholar
- Wainwright, M. J., & Jordan M. I. Graphical models, exponential families, and variational inference. Technical report, UC Berkeley, Department of Statistics, No. 649, September 2003.Google Scholar
- Wainwright, M. J., Jaakkola, T. S., Willsky, A. S. Tree-based reparameterization framework for analysis of sum-product and related algorithms. IEEE Trans. Info. Theory, 49(5):1120--1146, May 2003. Google ScholarDigital Library
- Wainwright, M. J., Jaakkola, T. S., Willsky, A. S. Exact MAP estimates via agreement on (hyper)trees: Linear programming and message-passing. IEEE Trans. Information Theory, 51(11):3697--3717, November 2005. Google ScholarDigital Library
- Weiss, Y., Yanover, C., Meltzer T. Map estimation, linear programming and belief propagation with convex free energies. In Uncertainty in Artificial Intelligence, 2007.Google Scholar
- Zhou, H. Long range frustration in a spin-glass model of the vertex cover problem. Physical Review Letters 94: 217203, 2005.Google ScholarCross Ref
Index Terms
- Reparameterization based consistent graph-structured linear programs
Recommendations
Smoothing-type algorithm for solving linear programs by using an augmented complementarity problem
We present a smoothing-type algorithm for solving the linear program (LP) by making use of an augmented system of its optimality conditions. The algorithm is shown to be globally convergent without requiring any assumption. It only needs to solve one ...
Relaxation Method for Large Scale Linear Programming Using Decomposition
We propose a new decomposition method for large-scale linear programming. This method dualizes an arbitrary subset of the constraints and then maximizes the resulting dual functional by dual ascent. The ascent directions are chosen from a finite set and ...
Revised dantzig-wolfe decomposition for staircase-structured linear programs
Staircase structured linear programs arise naturally in the study of engineering economic systems. One general approach to solving such LP's is the technique of nested decomposition of the primal or dual problem. The research described in this paper ...
Comments