Elsevier

Neurocomputing

Volume 416, 27 November 2020, Pages 256-265
Neurocomputing

Nonnegative Matrix Factorization over Continuous Signals using Parametrizable Functions,☆☆

https://doi.org/10.1016/j.neucom.2019.11.109Get rights and content

Abstract

Nonnegative matrix factorization is a popular data analysis tool able to extract significant features from nonnegative data. We consider an extension of this problem to handle functional data, using parametrizable nonnegative functions such as polynomials or splines. Factorizing continuous signals using these parametrizable functions improves both the accuracy of the factorization and its smoothness. We introduce a new approach based on a generalization of the Hierarchical Alternating Least Squares algorithm. Our method obtains solutions whose accuracy is similar to that of existing approaches using polynomials or splines, while its computational cost increases moderately with the size of the input, making it attractive for large-scale datasets.

Introduction

Nonnegative matrix factorization (NMF) is a commonly used linear dimensionality reduction technique for nonnegative data. This method compresses data and is able to filter noise. It expresses data vectors using a part-based representation [1], [2] and extracts nonnegative characteristic features from the dataset. To do so, NMF takes a collection of nonnegative vectors, concatenated as the columns of input matrix YR+m×n,and tries to decompose each of them as a nonnegative linear combination (with coefficients in XR+r×n) of a few nonnegative basis vectors, contained in the columns of AR+m×r. Since an exact decomposition is in general not achievable, several cost functions can be considered to measure the accuracy of the approximation, such as the commonly used Frobenius distance:minAR+m×r,XR+r×nYAXF2

The factorization performed in NMF problems is in general not unique [3]. Therefore, a penalty term on the objective function is often added to take into account a priori knowledge on the data, such as sparsity [4], smoothness [1], orthogonality [5], etc.

NMF is a non-convex problem, and is NP-Hard [6]. However, it is convex with respect to A when X is fixed, and vice-versa. Moreover, as YAXF2=YXAF2,the problem is symmetric in A and X. Hence, many NMF algorithms proceed by (approximately) minimizing the problem alternatively on both matrices [7], including the popular hierarchical alternating least squares method which we describe next.

The hierarchical alternating least squares method [8] (HALS) is frequently used to obtain state-of-the art results for NMF [9], [10], and corresponds to Algorithm 1 below.

HALS updates the matrix A through its columns a: j. Columns are successively updated by optimizing the problem considering all the other variables as fixed. The optimal update for a column is obtained by computing the optimal solution of the unconstrained problem and projecting it over the set of nonnegative vectors [11]. Note that each component in the solution of the unconstrained problem is obtained in closed-form as the minimum of a univariate convex quadratic function. Performing an update of all columns leads to a better approximation of the optimal matrix A for X fixed.

NMF is often used to analyze continuous signals, such as spectral data [12], [13], [14]. Many of these signals can be well approximated using parametrizable functions like polynomials or splines. Hence, in this work, we tackle the situation where input data consists of continuous signals (possibly discretized), and seek to obtain an NMF-like factorization where the columns of matrix A will be replaced by continuous functions. Indeed, decomposing input data as linear combinations of functions instead of vectors will allow considering the input signals continuously, and not only at some discretization points, and a suitable choice of parametrizable functions will enforce some intrinsic features on the recovered data, such as smoothness in the case of polynomials or splines.

Section 2 introduces the functional NMF problem, and proposes a generalization of HALS to compute factorizations over parametrizable functions. Section 3 focuses on a crucial component of the new HALS algorithm, namely the projection operator over the set of nonnegative parametrizable functions. Section 4 describes existing approaches for the functional NMF problem. These are then compared to our approach in Section 5, where several numerical experiments, using both synthetic and real-world signals, allow us to assess the accuracy and speed of all algorithms.

Section snippets

NMF over parametrizable functions (F-NMF)

Consider a set Y={y1(t),,yn(t)}of n univariate functions defined over a common fixed interval [a, b]. The functional NMF problem (F-NMF) aims at recovering a dictionary A={a1(t),ar(t)}of r nonnegative functions over interval [a, b], and a nonnegative mixing matrix XR+r×n,which can provide an approximate linear description of the original data as follows:yi(t)j=1raj(t)xjit[a,b],suchthataj(t)0,xji01jr,1in.

Factorization rank r ≪ n is provided, as well as a set Fof parametrizable

Projection onto the set of nonnegative functions

To perform the B update in F-HALS we need to project its columns onto the set F+([a,b]),so that the functions used in the factorization remain nonnegative. This projection, which is performed on a vector of coefficients fRd,can be obtained as the solution of the following minimization problem: [f]F+([a,b])is the minimizer ofmingfgM2s.t.gF+([a,b])ming(fg)M(fg)s.t.gF+([a,b])mingL(fg)22s.t.gF+([a,b]).Note that matrix M (defined in Table 1) appearing in the objective defines the

Prior work on functional NMF

Before testing our proposed F-HALS algorithm we summarize existing work on the functional NMF problem. Several authors recently considered the F-NMF problem where input functions yi(t) are known at some observed discretization points (this data is collected in matrix Y), corresponding to what we called the sum cost. To tackle the problem, they work with an NMF-like factorization Y ≈ AX where columns in matrix A are forced to correspond to the discretization of parametrizable functions (meaning

Experimental results

In this section, we assess the performance of the F-HALS algorithm (Algorithm 2) through experiments both over synthetic and real signals. Our method is compared with standard HALS and some of the methods described in the previous section.

Unless stated otherwise, the input signals we use are created as Y=AX+Nwhere NRm×nis an additive Gaussian noise with a chosen signal-to-noise ratio (SNR). Matrix XRr×nis randomly generated using a normal distribution N(0,1)with negative values replaced by

Discussion and conclusion

Extending the NMF framework to handle polynomial or spline signals leads to functional NMF, which enables the recovery of smoother features and is less sensitive to noise when compared to standard NMF applied to discretized signals.

In this work we adapt the HALS algorithm to this extension. Our new F-HALS algorithm requires the ability to project over sets of nonnegative parametrizable functions, which is computationally feasible for polynomials and splines. Two cost functions can be

Declaration of competing inerest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

François Glineur received dual engineering degrees from Universit de Mons and CentraleSuplec in 1997, and a PhD in Applied Sciences from Universit de Mons in 2001. He visited Delft University of Technology and McMaster University as a post-doctoral researcher, then joined Universit catholique de Louvain where he is currently a professor of applied mathematics at the Engineering School, member of the Center for Operations Research and Econometrics and the Institute of Information and

References (27)

  • A. Cichocki et al.

    Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation

    (2009)
  • D.D. Lee et al.

    Learning the parts of objects by non-negative matrix factorization

    Nature

    (1999)
  • D. Donoho et al.

    When does non-negative matrix factorization give a correct decomposition into parts?

    Advances in Neural Information Processing Systems 16

    (2004)
  • P.O. Hoyer

    Non-negative matrix factorization with sparseness constraints

    Journal of machine learning research

    (2004)
  • S. Choi

    Algorithms for orthogonal nonnegative matrix factorization

    Neural Networks IJCNN

    (2008)
  • S.A. Vavasis

    On the complexity of nonnegative matrix factorization

    SIAM Journal on Optimization

    (2009)
  • H. Kim et al.

    Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method

    SIAM journal on matrix analysis and applications

    (2008)
  • A. Cichocki et al.

    Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization

    International Conference on Independent Component Analysis and Signal Separation

    (2007)
  • J. Feng et al.

    Evaluation of different algorithms of nonnegative matrix factorization in temporal psychovisual modulation

    IEEE Transactions on Circuits and Systems for Video Technology

    (2013)
  • E. Karahan et al.

    Tensor analysis and fusion of multimodal brain images

    Proceedings of the IEEE

    (2015)
  • J. Kim et al.

    Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework

    Journal of Global Optimization

    (2014)
  • A.M. Darsono et al.

    β-divergence nonnegative matrix factorization on biomedical blind source separation

    Journal of Telecommunication, Electronic and Computer Engineering (JTEC)

    (2017)
  • H. Kameoka et al.

    Constrained and regularized variants of non-negative matrix factorization incorporating music-specific constraints

    Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on

    (2012)
  • Cited by (6)

    François Glineur received dual engineering degrees from Universit de Mons and CentraleSuplec in 1997, and a PhD in Applied Sciences from Universit de Mons in 2001. He visited Delft University of Technology and McMaster University as a post-doctoral researcher, then joined Universit catholique de Louvain where he is currently a professor of applied mathematics at the Engineering School, member of the Center for Operations Research and Econometrics and the Institute of Information and Communication Technologies, Electronics and Applied Mathematics. His research interests focus on optimization models and methods (mainly convex optimization and algorithmic efficiency) and their engineering applications, as well as on nonnegative matrix factorization and applications to data analysis.

    Cécile Hautecoeur is a Ph.D. student in the ICTEAM institute of the Universit Catholique de Louvain. She received a master’s degree in applied mathematics in 2018 in the same university. Her research interests are nonnegative matrix factorization with a special focus on high dimensional or continuous data.

    This work was supported by the Fonds de la Recherche Scientifique - FNRS and the Fonds Wetenschappelijk Onderzoek - Vlaanderen under EOS Project no 30468160.

    ☆☆

    An implementation of the algorithm presented in this article, as well as code generating the figures are available on Code Ocean at url: https://codeocean.com/capsule/2082198/tree.

    View full text