Abstract
Manifold regularized techniques have been extensively exploited in unsupervised learning like matrix factorization whose performance is heavily affected by the underlying graph regularization. However, there exist no principled ways to select reasonable graphs under the matrix decomposition setting, particularly in multiple heterogeneous graph sources. In this paper, we deal with the issue of searching for the optimal linear combination space of multiple graphs under the low rank matrix approximation model. Specifically, efficient projection onto the probabilistic simplex is utilized to optimize the coefficient vector of graphs, resulting in the sparse pattern of coefficients. This attractive property of sparsity can be interpreted as a criterion for selecting graphs, i.e., identifying the most discriminative graphs and removing the noisy or irrelevant graphs, so as to boost the low rank decomposition performance. Experimental results over diverse popular image and web document corpora corroborate the effectiveness of our new model in terms of clusterings.
Similar content being viewed by others
Notes
Both Coil-20 and MNIST can be found at http://www.cad.zju.edu.cn/home/dengcai/Data/MLData.html. The USPS database is downloaded from http://www.cs.nyu.edu/~roweis/data.html.
Both TDT2 and Reuters-21578 are available at http://www.cad.zju.edu.cn/home/dengcai/Data/TextData.html. CiteSeer and Cora can be downloaded from http://www.cs.umd.edu/%7Esen/lbc-proj/LBC.html. Besides, BBC and BBC sport can be accessed under http://mlg.ucd.ie/datasets/bbc.html.
References
Cai D, He X, Han J, Huang T (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560
Cai X, Nie F, Cai W, Huang H (2013) Heterogeneous image features integration via multi-modal semi-supervised learning model. In: ICCV
Chen Y, Zhang J, Cai D, Liu W, He X (2013) Nonnegative local coordinate factorization for image representation. IEEE Trans Image Process 22(3):969–979
Cheng J, Leng C, Li P, Wang M, Lu H (2014) Semi-supervised multi-graph hashing for scalable similarity search. Comput Vis Image Underst 124(0):12–21
Ding C, Li T, Jordan M (2010) Convex and semi-nonnegative matrix factorizations. IEEE Trans Pattern Anal Mach Intell 32(1):45–55
Duchi J, Shalev-Shwartz S, Singer Y, Chandra T (2008) Efficient projections onto the ℓ 1-ball for learning in high dimensions. In: ICML
Gower JC, Dijksterhuis GB (2004) Procrustes problems. Oxford statistical science, vol 30. Oxford University Press, USA
Karasuyama M, Mamitsuka H (2013) Multiple graph label propagation by sparse integration. IEEE Trans Neural Netw Learn Syst 24(12):1999–2012
Kim M (2013) Accelerated max-margin multiple kernel learning. Appl Intell 38(1):45–57
Kuang D, Park H, Ding C H Q (2012) Symmetric nonnegative matrix factorization for graph clustering. In: SDM
Li P, Bu J, Chen C, He Z, Cai D (2013) Relational multimanifold coclustering. IEEE Trans Cybern PP(99):1–11
Li W J, Yeung D Y (2009) Relation regularized matrix factorization. In: IJCAI
Liu H, Wu Z, Li X, Cai D, Huang T (2012) Constrained nonnegative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34(7):1299–1311
Liu W, He J, Chang S F (2010) Large graph construction for scalable semi-supervised learning. In: ICML
Liu W, Wang J, Chang S F (2012) Robust and scalable graph-based semisupervised learning. Proc IEEE 100(9):2624–2638
Liu W, Wang J, Ji R, Jiang Y G, Chang S F (2012) Supervised hashing with kernels. In: CVPR
Liu X, He J, Lang B (2014) Multiple feature kernel hashing for large-scale visual search. Pattern Recog 47(2):748–757
Lu Z, Peng Y (2013) Exhaustive and efficient constraint propagation: a graph-based learning approach and its applications. Int J Comput Vis 103(3):306–325
Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416
Nesterov Y (2003) Introductory lectures on convex optimization: a basic course. Kluwer Academic Publishers
Saad Y (2003) Iterative methods for sparse linear systems, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia
Shu X, Gao Y, Lu H (2012) Efficient linear discriminant analysis with locality preserving for face recognition. Pattern Recog 45(5):1892–1898
Wang JJY, Bensmail H, Gao X (2013) Multiple graph regularized nonnegative matrix factorization. Pattern Recog 46(10): 2840–2847
Wang JY, Bensmail H, Gao X (2012) Multiple graph regularized protein domain ranking. BMC Bioinformatics 13(1):1–11
Wang M, Hua XS, Hong R, Tang J, Qi GJ, Song Y (2009) Unified video annotation via multigraph learning. IEEE Trans Circ Syst Video Technol 19(5):733–746
Wang M, Li H, Tao D, Lu K, Wu X (2012) Multimodal graph-based reranking for web image search. IEEE Trans Image Process 21(11):4649–4661
Xu B, Bu J, Chen C, Wang C, Cai D, He X (2014) EMR: a scalable graph-based ranking model for content-based image retrieval. IEEE Trans Knowl Data Eng 99 (PrePrints)
Xu C, Tao D, Geng B, Yang L, Hua X S (2012) Ensemble manifold regularization. IEEE Trans Pattern Anal Mach Intell 34(6):1227–1233
Yang YB, Li YN, Pan LY, Li N, He GN (2013) Image retrieval based on augmented relational graph representation. Appl Intell 38(4):489–501
Zhang D, Zhou Z H, Chen S (2007) Semi-supervised dimensionality reduction. In: SDM
Zhang Z, Zhao K (2013) Low-rank matrix approximation with manifold regularization. IEEE Trans Pattern Anal Mach Intell 35(7):1717–1729
Acknowledgments
The authors would like to thank the anonymous referees for their constructive suggestions and comments. The third author was partially supported by the Research Funds of Shanghai Municipal Science and Technology Commission Grant 12511502902 and the National Natural Science Foundation of China Grant 61375053.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tao, L., Ip, H.H.S., Wang, Y. et al. Low rank approximation with sparse integration of multiple manifolds for data representation. Appl Intell 42, 430–446 (2015). https://doi.org/10.1007/s10489-014-0600-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-014-0600-7