Dexian Wang
Skip Abstract Section
Abstract
Clustering is a popular research topic in the field of data mining, in which the clustering method based on non-negative matrix factorization (NMF) has been widely employed. However, in the update process of NMF, there is no learning rate to guide the update as well as the update depends on the data itself, which leads to slow convergence and low clustering accuracy. To solve these problems, a generalized deep learning clustering (GDLC) algorithm based on NMF is proposed in this article. Firstly, a nonlinear constrained NMF (NNMF) algorithm is constructed to achieve sequential updates of the elements in the matrix guided by the learning rate. Then, the gradient values corresponding to the element update are transformed into generalized weights and generalized biases, by inputting the elements as well as their corresponding generalized weights and generalized biases into the nonlinear activation function to construct the GDLC algorithm. In addition, for improving the understanding of the GDLC algorithm, its detailed inference procedure and algorithm design are provided. Finally, the experimental results on eight datasets show that the GDLC algorithm has efficient performance.
- [1] . 1998. Linear discriminant analysis-a brief tutorial. Institute for Signal and Information Processing 18, 1998 (1998), 1–8.Google Scholar
- [2] . 2010. Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 8 (2010), 1548–1560.Google Scholar
- [3] . 2022. A survey of deep nonnegative matrix factorization. Neurocomputing 491 (2022), 305–320.Google ScholarDigital Library
- [4] . 2021. Tri-regularized nonnegative matrix tri-factorization for co-clustering. Knowledge-Based Systems 226 (2021), 107101.Google ScholarCross Ref
- [5] . 2022. Graph regularized sparse non-negative matrix factorization for clustering. IEEE Transactions on Computational Social Systems (2022), 1–12.Google ScholarCross Ref
- [6] . 2022. Biased unconstrained non-negative matrix factorization for clustering. Knowledge-Based Systems 239 (2022), 108040.Google ScholarDigital Library
- [7] . 2006. Orthogonal nonnegative matrix t-factorizations for clustering. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 126–135.Google ScholarDigital Library
- [8] . 2012. Online nonnegative matrix factorization with robust stochastic approximation. IEEE Transactions on Neural Networks and Learning Systems 23, 7 (2012), 1087–1099.Google ScholarCross Ref
- [9] . 2019. Sparse deep nonnegative matrix factorization. Big Data Mining and Analytics 3, 1 (2019), 13–28.Google ScholarCross Ref
- [10] . 2005. The Organization of Behavior: A Neuropsychological Theory. Psychology Press.Google ScholarCross Ref
- [11] . 2021. NMF with feature relationship preservation penalty term for clustering problems. Pattern Recognition 112 (2021), 107814.Google ScholarCross Ref
- [12] . 2014. Robust manifold nonnegative matrix factorization. ACM Transactions on Knowledge Discovery from Data (TKDD) 8, 3 (2014), 1–21.Google ScholarDigital Library
- [13] . 2020. Regularized nonnegative matrix factorization with adaptive local structure learning. Neurocomputing 382 (2020), 196–209.Google ScholarDigital Library
- [14] . 2013. A tutorial overview of vector and matrix norms. University of California, Berkeley, CA, Lecture Notes (2013), 19.Google Scholar
- [15] . 2015. Deep NMF for speech separation. In Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’15). IEEE, 66–70.Google Scholar
- [16] . 2015. Deep learning. Nature 521, 7553 (2015), 436–444.Google ScholarCross Ref
- [17] . 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788–791.Google ScholarCross Ref
- [18] . 2020. Relational completion based non-negative matrix factorization for predicting metabolite-disease associations. Knowledge-Based Systems 204 (2020), 106238.Google ScholarCross Ref
- [19] . 2015. Two efficient algorithms for approximately orthogonal nonnegative matrix factorization. IEEE Signal Processing Letters 22, 7 (2015), 843–846.Google ScholarCross Ref
- [20] . 2020. Deep nonsmooth nonnegative matrix factorization network with semi-supervised learning for SAR image change detection. ISPRS Journal of Photogrammetry and Remote Sensing 160 (2020), 167–179.Google ScholarCross Ref
- [21] . 2017. Graph regularized non-negative low-rank matrix factorization for image clustering. IEEE Transactions on Cybernetics 47, 11 (2017), 3840–3853.Google ScholarCross Ref
- [22] . 2018. Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints. Engineering Applications of Artificial Intelligence 69 (2018), 24–35.Google ScholarCross Ref
- [23] . 2019. K-multiple-means: A multiple-means clustering method with specified k clusters. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 959–967.Google ScholarDigital Library
- [24] . 2021. Robust semi-supervised nonnegative matrix factorization for image clustering. Pattern Recognition 111 (2021), 107683.Google ScholarCross Ref
- [25] . 1971. A convergence theorem for non negative almost supermartingales and some applications. In Proceedings of the Optimizing Methods in Statistics. Elsevier, 233–257.Google Scholar
- [26] . 1958. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65, 6 (1958), 386–408.Google ScholarCross Ref
- [27] . 1986. Learning representations by back-propagating errors. Nature 323, 6088 (1986), 533–536.Google ScholarCross Ref
- [28] . 2017. A review of clustering techniques and developments. Neurocomputing 267 (2017), 664–681.Google ScholarDigital Library
- [29] . 2012. Graph dual regularization non-negative matrix factorization for co-clustering. Pattern Recognition 45, 6 (2012), 2237–2250.Google ScholarDigital Library
- [30] . 2017. Non-negative spectral learning and sparse regression-based dual-graph regularized feature selection. IEEE Transactions on Cybernetics 48, 2 (2017), 793–806.Google ScholarCross Ref
- [31] . 2021. Continual multiview task learning via deep matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 32, 1 (2021), 139–150.Google ScholarCross Ref
- [32] . 2018. Sparse dual graph-regularized NMF for image co-clustering. Neurocomputing 316 (2018), 156–165.Google ScholarCross Ref
- [33] . 2005. Generalized principal component analysis (GPCA). IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 12 (2005), 1945–1959.Google ScholarDigital Library
- [34] . 2016. Orthogonal nonnegative matrix factorization combining multiple features for spectral–spatial dimensionality reduction of hyperspectral imagery. IEEE Transactions on Geoscience and Remote Sensing 54, 7 (2016), 4272–4286.Google ScholarCross Ref
- [35] . 1987. Principal component analysis. Chemometrics and Intelligent Laboratory Systems 2, 1-3 (1987), 37–52.Google ScholarCross Ref
- [36] . 2005. Survey of clustering algorithms. IEEE Transactions on Neural Networks 16, 3 (2005), 645–678.Google ScholarDigital Library
- [37] . 2021. Orthogonal nonnegative matrix factorization using a novel deep autoencoder network. Knowledge-Based Systems 227 (2021), 107236.Google ScholarDigital Library
- [38] . 2021. Uniform distribution non-negative matrix factorization for multiview clustering. IEEE Transactions on Cybernetics 51, 6 (2021), 3249–3262.Google ScholarCross Ref
- [39] . 2018. Deep autoencoder-like nonnegative matrix factorization for community detection. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 1393–1402.Google ScholarDigital Library
- [40] . 2014. Dual-graph regularized concept factorization for clustering. Neurocomputing 138 (2014), 120–130.Google ScholarCross Ref
- [41] . 2020. Joint feature representation and classification via adaptive graph semi-supervised nonnegative matrix factorization. Signal Processing: Image Communication 89 (2020), 115984.Google ScholarCross Ref
- [42] . 2016. Online nonnegative matrix factorization with outliers. IEEE Transactions on Signal Processing 65, 3 (2016), 555–570.Google ScholarDigital Library
- [43] . 2019. Deep non-negative matrix factorization architecture based on underlying basis images learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 6 (2019), 1897–1913.Google ScholarCross Ref
Index Terms
- A Generalized Deep Learning Clustering Algorithm Based on Non-Negative Matrix Factorization
Recommendations
Document Clustering Based on Spectral Clustering and Non-negative Matrix Factorization
IEA/AIE '08: Proceedings of the 21st international conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems: New Frontiers in Applied Artificial IntelligenceIn this paper, we propose a novel non-negative matrix factorization (NMF) to the affinity matrix for document clustering, which enforces non-negativity and orthogonality constraints simultaneously. With the help of orthogonality constraints, this NMF ...
Similarity-based clustering by left-stochastic matrix factorization
For similarity-based clustering, we propose modeling the entries of a given similarity matrix as the inner products of the unknown cluster probabilities. To estimate the cluster probabilities from the given similarity matrix, we introduce a left-...
New SVD based initialization strategy for non-negative matrix factorization
We give a new method to determine the rank of the factorization for NMF algorithms.We propose a novel method SVD-NMF to enhance initialization for NMF.The compute process is cheap.Numerical results show that SVD-NMF convergent faster than NNDSVD and ...
Comments