Elsevier

Information Sciences

Volume 597, June 2022, Pages 125-143
Information Sciences

Improving nonnegative matrix factorization with advanced graph regularization

https://doi.org/10.1016/j.ins.2022.03.008Get rights and content

Highlights

  • A new regularizer is proposed based on a linear projection.

  • Two iterative update procedures are developed for minimizing the new objective function.

  • Various experiments verify the superiority of the proposed algorithm.

Abstract

Nonnegative Matrix Factorization (NMF) produces interpretable solutions for many applications including collaborative filtering. Typically, regularization is needed to address issues such as overfitting and interpretability, especially for collaborative filtering where the rating matrices are sparse. However, the existing regularizers are typically constructed from the factorization results instead of the rating matrices. Intuitively, we regard these existing regularizers as representing either user factors or item factors and anticipate that a more holistic regularizer could improve the effectiveness of NMF. To this end, we propose a graph regularizer based on a linear projection of the rating matrix, and call the resulting method: Linear Projection and Graph Regularized Nonnegative Matrix Factorization (LPGNMF). We develop two iterative methods to minimize the cost function and derive two update rules named LPGNMF and F-LPGNMF. Additionally, we prove the value of the objective function decreases with LPGNMF and converges to a fixed point with F-LPGNMF. Finally, we test these methods against a number of NMF algorithms on different data sets and show both LPGNMF and F-LPGNMF always achieve smaller errors based on two different error measures.

Section snippets

1 Introduction

A recommendation system is a vital component of the modern Internet environment. It can help people locate their favorite items and filter the desired information out of a huge amount of data [5], [29]. Collaborative Filtering (CF) [1], [2], [19], [34] is one of the most popular methods for recommendation systems, especially for the large scale systems including movies [4], music [21], television [3], books [32], e-learning [5] or e-commerce [8]. Assuming users who liked similar items in the

2 Preliminaries

In this section, we briefly introduce the basic definitions of NMF and GNMF and their update rules for factorization. Before introducing these methods, we provide some notations that will be used throughout in this paper.

Suppose that we have a rating matrix X consisting of M users and N items (or movies). Let X=[x·1,x·2,,x·N]RM×N be the rating matrix (see Table 1), where x·j(j=1,2,,N) is the jth column of X. Indeed, x·j=(x1j,x2j,,xMj)T is the rating vector for item j from all users, and xij=

3. Linear Projection and Graph Regularized Nonnegative Matrix Factorization

NMF can learn a parts-based representation in the Euclidean space, but it is unable to unveil the inherent geometric structure of the original data space, which is important for practical applications. Although GNMF solved this problem by introducing a new regularizer with geometric structure, it still doesn’t involve the observed ratings of items nor the geometric structure of its projection matrix. In contrast, LPGNMF fuses three elements, user-item ratings matrix, item matrix and user

Experiments

In this section, we report several experiments to evaluate the performance of LPGNMF and F-LPGNMF and compare them with the existing approaches on eight data sets: Movielens 100 K, MovieLens 1 M, MovieLens 10 M, MovieLens-Latest-Small and Yelp data set (Pets, Active Life, Restaurants, and Beauty and Spas). The compared approaches include SR1VSS, SR2VSS, PRM2, NMF [24], [33], GNMF and GUNMF [6].

Performance comparison

We compare the performance of LPGNMF and F-LPGNMF with the existing methods including SR1VSS, SR2VSS [30], PRM2 [35], NMF [24], [33], GNMF and GUNMF [6] on two data sets: MovieLens and Yelp. Since the training set and testing set are divided randomly, and the setting of initial values of latent matrices is random, the results may slightly differ at each time. To solve this problem and reduce experimental error, we have repeated this procedure 20 times and taken their mean as the following

Conclusion

In this paper, we present a novel regularizer for nonnegative matrix factorization based on linear projection and geometric structure. We develop two iterative update formulas to minimize the proposed objective function, one is proposed based on multiplicative update rules and called LPGNMF, and another one is based alternative direction principle and called F-LPGNMF. The test results demonstrate LPGNMF and F-LPGNMF have a better prediction performance for movie ratings than SR1VSS, SR2VSS,

CRediT authorship contribution statement

Xiaoxia Zhang: Supervision, Investigation, Conceptualization, Methodology, Software, Writing - original draft, Validation. Degang Chen: Data curation, Resources, Formal analysis, Investigation. Hong Yu: Funding acquisition. Guoyin Wang: Funding acquisition. Kesheng Wu: Writing - review & editing, Visualization, Investigation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

The authors would like to thank the National Natural Science Fund of China (61876027), the State Key Program of National Nature Science Foundation of China (61936001), the Science and Technology Research Program of Chongqing Education Commission of China (KJQN201900638) and the National Energy Research Scientific Computing Center (NERSC).

References (41)

  • D. Cai et al.

    Graph regularized nonnegative matrix factorization for data representation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2011)
  • T.Y. Zhang et al.

    LPGNMF: predicting long non-coding rna and protein interaction using graph regularized nonnegative matrix factorization

    IEEE/ACM Trans. Comput. Biol. Bioinform.

    (2018)
  • J.J. Castro-Sanchez et al.

    A highly adaptive recommender system based on fuzzy logic for b2c e-commerce portals

    Expert Syst. Appl.

    (2011)
  • G. Chen et al.

    Collaborative filtering using orthogonal nonnegative matrix tri-factorization, Data Mining Workshops, 2007 ICDM Workshops 2007

    Seventh IEEE International Conference on IEEE

    (2007)
  • F.R.K. Chung

    Spectral Graph Theory

    Am. Math. Soc.

    (1997)
  • A.P. Dempster et al.

    Maximum likelihood from incomplete data via the Em algorithm

    J. R. Stat. Soc. Ser. B (Methodological)

    (1977)
  • C. Ding et al.

    Convex and semi-nonnegative matrix-factorizations

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2010)
  • P. Gopalan et al.

    Scalable recommendations with poisson factorization

    Eprint Arxiv

    (2013)
  • N. Guan et al.

    Online nonnegative matrix factorization with robust stochastic approximation

    IEEE Trans. Neural Networks Learn. Syst.

    (2012)
  • E.F. Gonzales et al.

    Accelerating the Lee-Seung algorithm for non-negative matrix factorization

    (2005)
  • Cited by (0)

    This work is jointly supported by the National Key R & D Program of China (2019YFB2103000), the National Natural Science Foundation of China (61936001, 62136002, 61876027), the Natural Science Foundation of Chongqing (cstc2019jcyj-cxttX0002, cstc2020jcyj-msxmX0737, cstc2021ycjh-bgzxm0013), the Key Cooperation Project of Chongqing Municipal Education Commission (HZ2021008), the Science and Technology Research Program of Chongqing Education Commission of China (KJQN201900638), the National Energy Research Scientific Computing Center (NERSC).

    View full text