research-article

Solution stability in linear programming relaxations: graph partitioning and unsupervised learning

Authors:

Sebastian Nowozin,

Stefanie JegelkaAuthors Info & Claims

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

Pages 769 - 776

https://doi.org/10.1145/1553374.1553473

Published: 14 June 2009 Publication History

Abstract

We propose a new method to quantify the solution stability of a large class of combinatorial optimization problems arising in machine learning. As practical example we apply the method to correlation clustering, clustering aggregation, modularity clustering, and relative performance significance clustering. Our method is extensively motivated by the idea of linear programming relaxations. We prove that when a relaxation is used to solve the original clustering problem, then the solution stability calculated by our method is conservative, that is, it never overestimates the solution stability of the true, unrelaxed problem. We also demonstrate how our method can be used to compute the entire path of optimal solutions as the optimization problem is increasingly perturbed. Experimentally, our method is shown to perform well on a number of benchmark problems.

References

[1]

Bansal, N., Blum, A., & Chawla, S. (2002). Correlation clustering. Foundations of Computer Science (FOCS'2002) (pp. 238--247). IEEE.

Digital Library

[2]

Bertsimas, D., & Tsitsiklis, J. N. (1997). Introduction to linear optimization. Athena Scientific, Massachusetts.

Digital Library

[3]

Brandes, U., Delling, D., Gaertler, M., Görke, R., Hoefer, M., Nikoloski, Z., & Wagner, D. (2008). On modularity clustering. IEEE Trans. Knowl. Data Eng, 20, 172--188.

Digital Library

[4]

Chopra, S., & Rao, M. R. (1993). The partition problem. Math. Program, 59, 87--115.

Digital Library

[5]

Demaine, E. D., Emanuel, D., Fiat, A., & Immorlica, N. (2006). Correlation clustering in general weighted graphs. Theor. Comput. Sci, 361, 172--187.

Digital Library

[6]

Deza, M. M., Grötschel, M., & Laurent, M. (1992). Clique-web facets for multicut polytopes. Mathematics of Operations Research, 17, 981--1000.

Digital Library

[7]

Deza, M. M., & Laurent, M. (1997). Geometry of cuts and metrics. Springer.

[8]

Emanuel, D., & Fiat, A. (2003). Correlation clustering -- minimizing disagreements on arbitrary weighted graphs. Eur. Symp. Alg. (pp. 208--220). Springer.

[9]

Finley, T., & Joachims, T. (2005). Supervised clustering with support vector machines. Intl. Conf. Mach. Learn. (pp. 217--224). ACM.

Digital Library

[10]

Gaertler, M., Görke, R., & Wagner, D. (2007). Significance-driven graph clustering. Algorithmic Aspects in Information and Management. Springer.

Digital Library

[11]

Gionis, A., Mannila, H., & Tsaparas, P. (2007). Clustering aggregation. Trans. on Know. Discovery from Data, 1.

Digital Library

[12]

Grötschel, M., & Wakabayashi, Y. (1989). A cutting plane algorithm for a clustering problem. Math. Prog., 45, 59--96.

Digital Library

[13]

Grötschel, M., & Wakabayashi, Y. (1990). Facets of the clique partitioning polytope. Math. Prog., 47.

Digital Library

[14]

Jansen, B., Jong, J., Roos, C., & Terlaky, T. (1997). Sensitivity analysis in linear programming: Just be careful! European Journal of Operational Research, 101, 15--28.

[15]

Joachims, T., & Hopcroft, J. E. (2005). Error bounds for correlation clustering. Intl. Conf. Mach. Learn. (pp. 385--392). ACM.

Digital Library

[16]

Kernighan, B. W., & Lin, S. (1970). An efficient heuristic procedure for partitioning graphs. The Bell System Technical Journal, 291--307.

[17]

Kilinç-Karzan, F., Toriello, A., Ahmed, S., Nemhauser, G., & Savelsbergh, M. (2007). Approximating the stability region for binary mixed-integer programs (Technical Report). Gatech.

[18]

Newman, M. (2004). Analysis of weighted networks (Technical Report). Cornell University.

[19]

Newman, M. E. J., & Girvan, M. (2004). Finding and evaluating community structure in networks. Physical Review E, 69.

[20]

Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. American Statistical Association Journal, 66, 846--850.

[21]

Schrijver, A. (1998). Theory of linear and integer programming. John Wiley & Sons.

Digital Library

[22]

Wolsey, L. A. (1998). Integer programming. John Wiley.

Cited By

Li ZYang XLiu JHong BZhang YZhai HShen LChen XLiu ZHan H(2024)DeepMulticut: Deep Learning of Multicut Problem for Neuron Segmentation From Electron Microscopy VolumeIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.340963446:12(8696-8714)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3409634
Arınık NFigueiredo RLabatut V(2023)Efficient enumeration of the optimal solutions to the correlation clustering problemJournal of Global Optimization10.1007/s10898-023-01270-386:2(355-391)Online publication date: 16-Feb-2023
https://doi.org/10.1007/s10898-023-01270-3
Abbas ASwoboda P(2022)RAMA: A Rapid Multicut Algorithm on GPU2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00802(8183-8192)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00802
Show More Cited By

Index Terms

Solution stability in linear programming relaxations: graph partitioning and unsupervised learning

Recommendations

On linear programming relaxations for solving polynomial programming problems
Highlights
- Theoretically compared the tightness of the J-set relaxation with the recursive McCormick relaxations.
Abstract
This paper studies linear programming (LP) relaxations for solving polynomial programming problems. A polynomial programming problem can be equivalently formulated as a quadratically constrained quadratic program (QCQP) by introducing ...
On asymptotic stability of linear time-varying systems

This paper is concerned with asymptotic stability analysis of linear time-varying (LTV) systems. With the help of the notion of stable functions, some differential Lyapunov inequalities (DLIs) based necessary and sufficient conditions are derived for ...
Semidefinite relaxations for quadratically constrained quadratic programming: A review and comparisons

At the intersection of nonlinear and combinatorial optimization, quadratic programming has attracted significant interest over the past several decades. A variety of relaxations for quadratically constrained quadratic programming (QCQP) can be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

June 2009

1331 pages

ISBN:9781605585161

DOI:10.1145/1553374

General Chair:
Andrea Danyluk
Williams College
,
Program Chairs:
Léon Bottou
NEC Laboratories America
,
Michael Littman
Rutgers University

Copyright © 2009 Copyright 2009 by the author(s)/owner(s).

Sponsors

NSF
Microsoft Research: Microsoft Research
MITACS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Sixth Framework Programme

Conference

ICML '09

Sponsor:

Microsoft Research

ICML '09: The 26th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 14 - 18, 2009

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
284
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li ZYang XLiu JHong BZhang YZhai HShen LChen XLiu ZHan H(2024)DeepMulticut: Deep Learning of Multicut Problem for Neuron Segmentation From Electron Microscopy VolumeIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.340963446:12(8696-8714)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3409634
Arınık NFigueiredo RLabatut V(2023)Efficient enumeration of the optimal solutions to the correlation clustering problemJournal of Global Optimization10.1007/s10898-023-01270-386:2(355-391)Online publication date: 16-Feb-2023
https://doi.org/10.1007/s10898-023-01270-3
Abbas ASwoboda P(2022)RAMA: A Rapid Multicut Algorithm on GPU2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00802(8183-8192)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00802
Usmani UHapponen AWatada J(2022)Enhanced Deep Learning Framework for Fine-Grained Segmentation of Fashion and ApparelIntelligent Computing10.1007/978-3-031-10464-0_3(29-44)Online publication date: 7-Jul-2022
https://doi.org/10.1007/978-3-031-10464-0_3
Lukasik JKeuper MSingh MYarkony J(2020)A Benders Decomposition Approach to Correlation Clustering2020 IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments (MLHPC) and Workshop on Artificial Intelligence and Machine Learning for Scientific Applications (AI4S)10.1109/MLHPCAI4S51975.2020.00009(9-16)Online publication date: Nov-2020
https://doi.org/10.1109/MLHPCAI4S51975.2020.00009
Song JAndres BBlack MHilliges OTang S(2019)End-to-End Learning for Graph Decomposition2019 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV.2019.01019(10092-10101)Online publication date: Oct-2019
https://doi.org/10.1109/ICCV.2019.01019
Kardoost AKeuper M(2019)Solving Minimum Cost Lifted Multicut Problems by Node AgglomerationComputer Vision – ACCV 201810.1007/978-3-030-20870-7_5(74-89)Online publication date: 25-May-2019
https://doi.org/10.1007/978-3-030-20870-7_5
Horňáková ALange JAndres B(2017)Analysis and optimization of graph decompositions by lifted multicutsProceedings of the 34th International Conference on Machine Learning - Volume 7010.5555/3305381.3305540(1539-1548)Online publication date: 6-Aug-2017
https://dl.acm.org/doi/10.5555/3305381.3305540
Swoboda PAndres B(2017)A Message Passing Algorithm for the Minimum Cost Multicut Problem2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR.2017.530(4990-4999)Online publication date: Jul-2017
https://doi.org/10.1109/CVPR.2017.530
Levinkov EKirillov AAndres B(2017)A Comparative Study of Local Search Algorithms for Correlation ClusteringPattern Recognition10.1007/978-3-319-66709-6_9(103-114)Online publication date: 15-Aug-2017
https://doi.org/10.1007/978-3-319-66709-6_9
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten