Abstract
Reverse-convex programming (RCP) concerns global optimization of a specific class of non-convex optimization problems. We show that a recently proposed model for sparse non-negative matrix factorization (NMF) belongs to this class. Based on this result, we design two algorithms for sparse NMF that solve sequences of convex second-order cone programs (SOCP).
We work out some well-defined modifications of NMF that leave the original model invariant from the optimization viewpoint. They considerably generalize the sparse NMF setting to account for uncertainty in sparseness, for supervised learning, and, by dropping the non-negativity constraint, for sparsity-controlled PCA.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Horst, R., Tuy, H.: Global Optimization. Springer, Berlin (1996)
Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. of Mach. Learning Res. 5, 1457–1469 (2004)
Shen, J., Israël, G.W.: A receptor model using a specific non-negative transformation technique for ambient aerosol. Atmospheric Environment 23(10), 2289–2298 (1989)
Paatero, P., Tapper, U.: Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5, 111–126 (1994)
Xu, W., Liu, X., Gong, Y.: Document clustering based on non-negative matrix factorization. In: SIGIR 2003: Proc. of the 26th Ann. Intl. ACM SIGIR Conf. on Res. and Developm. in Info. Retrieval, pp. 267–273. ACM Press, New York (2003)
Hoyer, P.O., Hyvärinen, A.: A multi-layer sparse coding network learns contour coding from natural images. Vision Research 42(12), 1593–1605 (2002)
Smaragdis, P., Brown, J.C.: Non-negative matrix factorization for polyphonic music transcription. In: IEEE Workshop on Appl. of Sign. Proc. to Audio and Acoustics, pp. 177–180 (2003)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)
Donoho, D., Stodden, V.: When does non-negative matrix factorization give a correct decomposition into parts? In: Adv. in NIPS, vol. 17 (2004)
Li, S.Z., Hou, X.W., Zhang, H.J., Cheng, Q.S.: Learning spatially localized, parts-based representation. In: Proc. of CVPR (2001)
Wang, Y., Jia, Y., Hu, C., Turk, M.: Fisher non-negative matrix factorization for learning local features. In: Proc. Asian Conf. on Comp. Vision (2004)
Littlestone, N., Warmuth, M.: Relating data compression, learnability, and the Vapnik-Chervonenkis dimension. Tech. Rep., Univ. of Calif. Santa Cruz (1986)
Herbrich, R., Williamson, R.C.: Algorithmic luckiness. J. of Mach. Learning Res. 3, 175–212 (2002)
Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37, 3311–3325 (1997)
Sturm, J.F.: Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones (updated version 1.05). Department of Econometrics, Tilburg University, Tilburg, The Netherlands (2001)
ApS, M. (ed.): The MOSEK optimization tools version 3.2 (Revision 8) User’s manual and reference, MOSEK ApS, Denmark (2005)
Lobo, M.S., Vandenberghe, L., Boyd, S., Lebret, H.: Applications of second-order cone programming. Linear Algebra and its Applications (1998)
Tuy, H.: Convex programs with an additional reverse convex constraint. J. of Optim. Theory and Applic. 52, 463–486 (1987)
Horst, R., Pardalos, P.M. (eds.): Handbook of Global Optimization. Kluwer Academic Publishers, Dordrecht (1995)
d’Aspremont, A., Ghaoui, L.E., Jordan, M.I., Lanckriet, G.R.: A direct formulation for sparse PCA using semidefinite programming. In: Adv. in NIPS (2004)
Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. of Comp. a. Graph. Statistics (to appear)
Chennubholta, C., Jepson, A.: Sparse PCA extracting multi-scale structure from data. In: Proc. of ICCV, pp. 641–647 (2001)
Paatero, P.: Least squares formulation of robust non-negative factor analysis. Chemometrics and Intelligent Laboratory Systems 37 (1997)
CBCL, CBCL face database #1. MIT Center For Biological and Computational Learning (2000), http://cbcl.mit.edu/software-datasets
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Heiler, M., Schnörr, C. (2005). Reverse-Convex Programming for Sparse Image Codes. In: Rangarajan, A., Vemuri, B., Yuille, A.L. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2005. Lecture Notes in Computer Science, vol 3757. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11585978_39
Download citation
DOI: https://doi.org/10.1007/11585978_39
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-30287-2
Online ISBN: 978-3-540-32098-2
eBook Packages: Computer ScienceComputer Science (R0)