research-article

Refined-Graph Regularization-Based Nonnegative Matrix Factorization

Authors:

Yongsheng DongAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 9, Issue 1

Article No.: 1, Pages 1 - 21

https://doi.org/10.1145/3090312

Published: 21 August 2017 Publication History

Abstract

Nonnegative matrix factorization (NMF) is one of the most popular data representation methods in the field of computer vision and pattern recognition. High-dimension data are usually assumed to be sampled from the submanifold embedded in the original high-dimension space. To preserve the locality geometric structure of the data, k-nearest neighbor (k-NN) graph is often constructed to encode the near-neighbor layout structure. However, k-NN graph is based on Euclidean distance, which is sensitive to noise and outliers. In this article, we propose a refined-graph regularized nonnegative matrix factorization by employing a manifold regularized least-squares regression (MRLSR) method to compute the refined graph. In particular, each sample is represented by the whole dataset regularized with ℓ₂-norm and Laplacian regularizer. Then a MRLSR graph is constructed based on the representative coefficients of each sample. Moreover, we present two optimization schemes to generate refined-graphs by employing a hard-thresholding technique. We further propose two refined-graph regularized nonnegative matrix factorization methods and use them to perform image clustering. Experimental results on several image datasets reveal that they outperform 11 representative methods.

References

[1]

Shounan An, Jiho Yoo, and Seungjin Choi. 2011. Manifold-respecting discriminant nonnegative matrix factorization. Pattern Recogn. Lett. 32, 6 (2011), 832--837

Digital Library

[2]

Mikhail Belkin and Partha Niyogi. 2001. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems. 585--591.

Digital Library

[3]

Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Nov. 2006. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7 (Nov. 2006), 2399--2434.

Digital Library

[4]

Deng Cai, Xiaofei He, Jiawei Han, and Thomas S. Huang. Aug. 2011. Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33 (Aug. 2011), 1548--1560.

Digital Library

[5]

Lin Chen, Dong Xu, Ivor Wai-Hung Tsang, and Xuelong Li. Jul. 2014. Spectral embedded hashing for scalable image retrieval. IEEE Trans. Cybernet. 44, 7 (Jul. 2014), 1180--1190.

[6]

Fan R. K. Chung. 1997. Spectral Graph Theory. Amer. Math. Soc.

[7]

Yongsheng Dong, Dacheng Tao, and Xuelong Li. Oct. 2015. Nonnegative multiresolution representation-based texture image classification. ACM Trans. Intell. Syst. Technol. 7, 1 (Oct. 2015), 4.

Digital Library

[8]

Ehsan Elhamifar and René Vidal. Jun. 2009. Sparse subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09). IEEE, 2790--2797.

[9]

Charles Elkan. Aug. 2003. Using the triangle inequality to accelerate k-means. In Proceedings of the 20th International Conference on Machine Learning (ICML’03), Vol. 3. 147--153.

Digital Library

[10]

Hongchang Gao, Feiping Nie, Weidong Cai, and Heng Huang. 2015. Robust capped norm nonnegative matrix factorization: Capped norm NMF. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 871--880.

Digital Library

[11]

Lianli Gao, Jingkuan Song, Feiping Nie, Fuhao Zou, Nicu Sebe, and Heng Tao Shen. 2016. Graph-without-cut: An ideal graph learning for image segmentation. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16). AAAI Press, 1188--1194.

Digital Library

[12]

Quanquan Gu and Jie Zhou. 2009. Neighborhood preserving nonnegative matrix factorization. In Proceedings of the British Machine Vision Conference (BMVC’09). 1--10.

[13]

Xiaofei He, Deng Cai, Shuicheng Yan, and Hong-Jiang Zhang. 2005. Neighborhood preserving embedding. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05). IEEE, 1208--1213.

Digital Library

[14]

Patrik O. Hoyer. 2002. Non-negative sparse coding. In Proceedings of the IEEE Workshop Neural Networks for Signal Processing. IEEE, 557--565.

[15]

Jin Huang, Feiping Nie, Heng Huang, and Chris Ding. 2014. Robust manifold nonnegative matrix factorization. ACM Trans. Knowl. Discov. Data (TKDD) 8, 3 (2014), 11.

Digital Library

[16]

Aapo Hyvärinen, Juha Karhunen, and Erkki Oja. 2004. Independent Component Analysis. Vol. 46. John Wiley 8 Sons.

[17]

Alan Julian Izenman. 2013. Linear discriminant analysis. In Modern Multivariate Statistical Techniques. Springer, 237--280.

[18]

Ian Jolliffe. 2005. Principal Component Analysis. Wiley Online Library.

[19]

Daniel D. Lee and H. Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788--791.

[20]

Daniel D. Lee and H. Sebastian Seung. 2000. Algorithms for non-negative matrix factorization. In Advances in Neural Information Processing Systems. 556--562.

[21]

Stan Z. Li, XinWen Hou, HongJiang Zhang, and QianSheng Cheng. Dec. 2001. Learning spatially localized, parts-based representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’01), Vol. 1. 1--207.

[22]

Xuelong Li, Guosheng Cui, and Yongsheng Dong. 2016. Graph regularized non-negative low-rank matrix factorization for image clustering. IEEE Trans. Cybernet. PP, 99 (2016), 1--14.

[23]

Xuelong Li, Jianhua Shi, Yongsheng Dong, and Dacheng Tao. 2015. A survey on scene image classification. Sci. China Info. Sci. 45, 7 (2015), 827--848.

[24]

Guangcan Liu, Zhouchen Lin, and Yong Yu. 2010. Robust subspace segmentation by low-rank representation. In Proceedings of the International Conference on Machine Learning (ICML’10). 663--670.

Digital Library

[25]

Haifeng Liu, Zhaohui Wu, Xuelong Li, Deng Cai, and Thomas S. Huang. July 2012. Constrained nonnegative matrix factorization for image representation. IEEE Trans. Pattern Anal.d Mach. Intell. 34, 7 (July 2012), 1299--1311.

Digital Library

[26]

Xiaobai Liu, Shuicheng Yan, and Hai Jin. May 2010. Projective nonnegative graph embedding. IEEE Trans. Image Process. 19, 5 (May 2010), 1126--1137.

Digital Library

[27]

Can-Yi Lu, Hai Min, Zhong-Qiu Zhao, Lin Zhu, De-Shuang Huang, and Shuicheng Yan. 2012. Robust and efficient subspace segmentation via least-squares regression. In Proceedings of the 12th European Conference on Computer Vision (ECCV’12). 347--360.

Digital Library

[28]

John Makhoul, Salim Roucos, and Herbert Gish. 1985. Vector quantization in speech coding. Proc. IEEE 73, 11 (1985), 1551--1588.

[29]

Feiping Nie, Wei Zhu, and Xuelong Li. 2017. Unsupervised large graph embedding. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2422--2428.

[30]

X. Niyogi. 2004. Locality preserving projections. In Advances in Neural Information Processing Systems. 153--160.

[31]

Christos H. Papadimitriou and Kenneth Steiglitz. 1982. Combinatorial Optimization: Algorithms and Complexity. Prentice-Hall.

Digital Library

[32]

Xi Peng, Zhang Yi, and Huajin Tang. 2015. Robust subspace clustering via thresholding ridge regression. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 3827--3833.

Digital Library

[33]

Guo-Jun Qi, Charu Aggarwal, Qi Tian, Heng Ji, and Thomas Huang. May 2012. Exploring context and content links in social media: A latent space method. IEEE Trans. Pattern Anal. Mach. Intell. 34, 5 (May 2012), 850--862.

Digital Library

[34]

Weiya Ren, Guohui Li, Dan Tu, and Li Jia. 2014. Nonnegative matrix factorization with regularizations. IEEE J. Emerg. Select. Top. Circ. Syst. 4, 1 (2014), 153--164.

[35]

Sam T. Roweis and Lawrence K. Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 5500 (2000), 2323--2326.

[36]

Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. 1998. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computat. 10, 5 (1998), 1299--1319.

Digital Library

[37]

Jingkuan Song, Lianli Gao, Feiping Nie, Heng Tao Shen, Yan Yan, and Nicu Sebe. Nov. 2016. Optimized graph learning using partial tags and multiple features for image and video annotation. IEEE Trans. Image Process. 25, 11 (Nov. 2016), 4999--5011.

Digital Library

[38]

Kun Song, Feiping Nie, Junwei Han, and Xuelong Li. 2017. Parameter free large margin nearest neighbor for distance metric learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2555--2561.

[39]

Alexander Strehl and Joydeep Ghosh. 2002. Cluster ensembles—A knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3 (2002), 583--617.

Digital Library

[40]

Dapeng Tao, Lianwen Jin, Weifeng Liu, and Xuelong LiJun. 2013. Hessian regularized support vector machines for mobile image annotation on the cloud. IEEE Trans. Multimed. 15, 4 (Jun. 2013), 833--844.

Digital Library

[41]

Joshua B. Tenenbaum, Vin De Silva, and John C. Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500 (2000), 2319--2323.

[42]

Yanhui Xiao, Zhenfeng Zhu, Yao Zhao, Yunchao Wei, Shikui Wei, and Xuelong Li. Oct. 2014. Topographic NMF for data representation. IEEE Trans. Cybernet. 44, 10 (Oct. 2014), 1762--1771.

[43]

Jianchao Yang, Shuicheng Yang, Yun Fu, Xuelong Li, and Thomas Huang. June 2008. Non-negative graph embedding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’08). IEEE, 1--8.

[44]

Yuan Yuan, Xuelong Li, Yanwei Pang, Xin Lu, and Dacheng Tao. May 2009. Binary sparse nonnegative matrix factorization. IEEE Trans. Circ. Syst. Video Technol. 19, 5 (May 2009), 772--777.

Digital Library

[45]

Stefanos Zafeiriou, Anastasios Tefas, Ioan Buciu, and Ioannis Pitas. May 2006. Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification. IEEE Trans. Neural Netw. 17, 3 (May 2006), 683--695.

Digital Library

[46]

Kun Zeng, Jun Yu, Cuihua Li, Jane You, and Taisong Jin. 2014. Image clustering by hyper-graph regularized non-negative matrix factorization. Neurocomputing 138 (2014), 209--217.

[47]

Liansheng Zhuang, Zihan Zhou, Jingwen Yin, Shenghua Gao, Zhouchen Lin, Yi Ma, and Nenghai Yu. 2016. Graph construction with label information for semi-supervised learning. arXiv:1607.02539 (2016).

Cited By

Liu JZhang DZhao JZheng CGao Y(2023)Non-Negative Low-Rank Representation With Similarity Correction for Cell Type Identification in scRNA-Seq DataIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2023.331937520:6(3737-3747)Online publication date: Nov-2023
https://doi.org/10.1109/TCBB.2023.3319375
Wang JWang LQiao TYuan S(2022)A Multi-Graph Laplacian Regularized Low-Rank Representation method for cancer sample clustering with integrated TCGA data2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM55620.2022.9995009(3179-3186)Online publication date: 6-Dec-2022
https://doi.org/10.1109/BIBM55620.2022.9995009
Feng PPan PZhou TChen HLuo CDemartini GZuccon GCulpepper JHuang ZTong H(2021)Zero Shot on the Cold-Start ProblemProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482312(474-483)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482312
Show More Cited By

Index Terms

Refined-Graph Regularization-Based Nonnegative Matrix Factorization
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Factorization methods
        Non-negative matrix factorization

Recommendations

Discriminative and Orthogonal Subspace Constraints-Based Nonnegative Matrix Factorization
Regular Papers

Nonnegative matrix factorization (NMF) is one widely used feature extraction technology in the tasks of image clustering and image classification. For the former task, various unsupervised NMF methods based on the data distribution structure information ...
Orthogonal Dual Graph Regularized Nonnegative Matrix Factorization
Image and Graphics
Abstract
Nonnegative matrix factorization (NMF) is a classical low-rank approximation method of data matrix, which decomposes a high-dimensional data matrix into two nonnegative low-rank matrices, namely basis matrix and coefficient matrix. In order to ...
Multiple graph regularized nonnegative matrix factorization

Non-negative matrix factorization (NMF) has been widely used as a data representation method based on components. To overcome the disadvantage of NMF in failing to consider the manifold structure of a data set, graph regularized NMF (GrNMF) has been ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 9, Issue 1

Regular Papers and Special Issue: Data-driven Intelligence for Wireless Networking

January 2018

258 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/3134224

Editor:
Yu Zheng
Microsoft Research, China

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 August 2017

Accepted: 01 May 2017

Revised: 01 April 2017

Received: 01 January 2017

Published in TIST Volume 9, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

State Key Laboratory of Virtual Reality Technology and Systems
National Natural Science Foundation of China
Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province
International Science and Technology Cooperation Project of Henan Province

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
352
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu JZhang DZhao JZheng CGao Y(2023)Non-Negative Low-Rank Representation With Similarity Correction for Cell Type Identification in scRNA-Seq DataIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2023.331937520:6(3737-3747)Online publication date: Nov-2023
https://doi.org/10.1109/TCBB.2023.3319375
Wang JWang LQiao TYuan S(2022)A Multi-Graph Laplacian Regularized Low-Rank Representation method for cancer sample clustering with integrated TCGA data2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM55620.2022.9995009(3179-3186)Online publication date: 6-Dec-2022
https://doi.org/10.1109/BIBM55620.2022.9995009
Feng PPan PZhou TChen HLuo CDemartini GZuccon GCulpepper JHuang ZTong H(2021)Zero Shot on the Cold-Start ProblemProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482312(474-483)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482312
Ji LQin QHan BYang HDemartini GZuccon GCulpepper JHuang ZTong H(2021)Reinforcement Learning to Optimize Lifetime Value in Cold-Start RecommendationProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482292(782-791)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482292
Zheng JMa QGu HZheng ZZhu FChin Ooi BMiao C(2021)Multi-view Denoising Graph Auto-Encoders on Heterogeneous Information Networks for Cold-start RecommendationProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3467427(2338-2348)Online publication date: 14-Aug-2021
https://dl.acm.org/doi/10.1145/3447548.3467427
Yu NWu MLiu JZheng CXu Y(2021)Correntropy-Based Hypergraph Regularized NMF for Clustering and Feature Selection on Multi-Cancer Integrated DataIEEE Transactions on Cybernetics10.1109/TCYB.2020.300079951:8(3952-3963)Online publication date: Aug-2021
https://doi.org/10.1109/TCYB.2020.3000799
Dong YJin MLi XMa JLiu ZWang LZheng L(2021)Compact Interchannel Sampling Difference Descriptor for Color Texture ClassificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2020.301452631:5(1684-1696)Online publication date: May-2021
https://doi.org/10.1109/TCSVT.2020.3014526
Dong MYuan FYao LXu XZhu LGupta RLiu YTang JPrakash B(2020)MAMOProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403113(688-697)Online publication date: 23-Aug-2020
https://dl.acm.org/doi/10.1145/3394486.3403113
Yeo JHwang SKim SKoh ELipka N(2020)Conversion Prediction from Clickstream: Modeling Market Prediction and Customer PredictabilityIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2018.288446732:2(246-259)Online publication date: 1-Feb-2020
https://doi.org/10.1109/TKDE.2018.2884467
Karimi MMajidi MMirSaeedi HArefi MOskuoee M(2020)A Novel Application of Deep Belief Networks in Learning Partial Discharge Patterns for Classifying Corona, Surface, and Internal DischargesIEEE Transactions on Industrial Electronics10.1109/TIE.2019.290858067:4(3277-3287)Online publication date: Apr-2020
https://doi.org/10.1109/TIE.2019.2908580
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents