Nonnegative Matrix Factorization over Continuous Signals using Parametrizable Functions☆,☆☆
Introduction
Nonnegative matrix factorization (NMF) is a commonly used linear dimensionality reduction technique for nonnegative data. This method compresses data and is able to filter noise. It expresses data vectors using a part-based representation [1], [2] and extracts nonnegative characteristic features from the dataset. To do so, NMF takes a collection of nonnegative vectors, concatenated as the columns of input matrix and tries to decompose each of them as a nonnegative linear combination (with coefficients in ) of a few nonnegative basis vectors, contained in the columns of . Since an exact decomposition is in general not achievable, several cost functions can be considered to measure the accuracy of the approximation, such as the commonly used Frobenius distance:
The factorization performed in NMF problems is in general not unique [3]. Therefore, a penalty term on the objective function is often added to take into account a priori knowledge on the data, such as sparsity [4], smoothness [1], orthogonality [5], etc.
NMF is a non-convex problem, and is NP-Hard [6]. However, it is convex with respect to A when X is fixed, and vice-versa. Moreover, as the problem is symmetric in A and X. Hence, many NMF algorithms proceed by (approximately) minimizing the problem alternatively on both matrices [7], including the popular hierarchical alternating least squares method which we describe next.
The hierarchical alternating least squares method [8] (HALS) is frequently used to obtain state-of-the art results for NMF [9], [10], and corresponds to Algorithm 1 below.
HALS updates the matrix A through its columns a: j. Columns are successively updated by optimizing the problem considering all the other variables as fixed. The optimal update for a column is obtained by computing the optimal solution of the unconstrained problem and projecting it over the set of nonnegative vectors [11]. Note that each component in the solution of the unconstrained problem is obtained in closed-form as the minimum of a univariate convex quadratic function. Performing an update of all columns leads to a better approximation of the optimal matrix A for X fixed.
NMF is often used to analyze continuous signals, such as spectral data [12], [13], [14]. Many of these signals can be well approximated using parametrizable functions like polynomials or splines. Hence, in this work, we tackle the situation where input data consists of continuous signals (possibly discretized), and seek to obtain an NMF-like factorization where the columns of matrix A will be replaced by continuous functions. Indeed, decomposing input data as linear combinations of functions instead of vectors will allow considering the input signals continuously, and not only at some discretization points, and a suitable choice of parametrizable functions will enforce some intrinsic features on the recovered data, such as smoothness in the case of polynomials or splines.
Section 2 introduces the functional NMF problem, and proposes a generalization of HALS to compute factorizations over parametrizable functions. Section 3 focuses on a crucial component of the new HALS algorithm, namely the projection operator over the set of nonnegative parametrizable functions. Section 4 describes existing approaches for the functional NMF problem. These are then compared to our approach in Section 5, where several numerical experiments, using both synthetic and real-world signals, allow us to assess the accuracy and speed of all algorithms.
Section snippets
NMF over parametrizable functions (F-NMF)
Consider a set of n univariate functions defined over a common fixed interval [a, b]. The functional NMF problem (F-NMF) aims at recovering a dictionary of r nonnegative functions over interval [a, b], and a nonnegative mixing matrix which can provide an approximate linear description of the original data as follows:
Factorization rank r ≪ n is provided, as well as a set of parametrizable
Projection onto the set of nonnegative functions
To perform the B update in F-HALS we need to project its columns onto the set so that the functions used in the factorization remain nonnegative. This projection, which is performed on a vector of coefficients can be obtained as the solution of the following minimization problem: is the minimizer ofNote that matrix M (defined in Table 1) appearing in the objective defines the
Prior work on functional NMF
Before testing our proposed F-HALS algorithm we summarize existing work on the functional NMF problem. Several authors recently considered the F-NMF problem where input functions yi(t) are known at some observed discretization points (this data is collected in matrix Y), corresponding to what we called the sum cost. To tackle the problem, they work with an NMF-like factorization Y ≈ AX where columns in matrix A are forced to correspond to the discretization of parametrizable functions (meaning
Experimental results
In this section, we assess the performance of the F-HALS algorithm (Algorithm 2) through experiments both over synthetic and real signals. Our method is compared with standard HALS and some of the methods described in the previous section.
Unless stated otherwise, the input signals we use are created as where is an additive Gaussian noise with a chosen signal-to-noise ratio (SNR). Matrix is randomly generated using a normal distribution with negative values replaced by
Discussion and conclusion
Extending the NMF framework to handle polynomial or spline signals leads to functional NMF, which enables the recovery of smoother features and is less sensitive to noise when compared to standard NMF applied to discretized signals.
In this work we adapt the HALS algorithm to this extension. Our new F-HALS algorithm requires the ability to project over sets of nonnegative parametrizable functions, which is computationally feasible for polynomials and splines. Two cost functions can be
Declaration of competing inerest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
François Glineur received dual engineering degrees from Universit de Mons and CentraleSuplec in 1997, and a PhD in Applied Sciences from Universit de Mons in 2001. He visited Delft University of Technology and McMaster University as a post-doctoral researcher, then joined Universit catholique de Louvain where he is currently a professor of applied mathematics at the Engineering School, member of the Center for Operations Research and Econometrics and the Institute of Information and
References (27)
- et al.
Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation
(2009) - et al.
Learning the parts of objects by non-negative matrix factorization
Nature
(1999) - et al.
When does non-negative matrix factorization give a correct decomposition into parts?
Advances in Neural Information Processing Systems 16
(2004) Non-negative matrix factorization with sparseness constraints
Journal of machine learning research
(2004)Algorithms for orthogonal nonnegative matrix factorization
Neural Networks IJCNN
(2008)On the complexity of nonnegative matrix factorization
SIAM Journal on Optimization
(2009)- et al.
Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method
SIAM journal on matrix analysis and applications
(2008) - et al.
Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization
International Conference on Independent Component Analysis and Signal Separation
(2007) - et al.
Evaluation of different algorithms of nonnegative matrix factorization in temporal psychovisual modulation
IEEE Transactions on Circuits and Systems for Video Technology
(2013) - et al.
Tensor analysis and fusion of multimodal brain images
Proceedings of the IEEE
(2015)
Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework
Journal of Global Optimization
β-divergence nonnegative matrix factorization on biomedical blind source separation
Journal of Telecommunication, Electronic and Computer Engineering (JTEC)
Constrained and regularized variants of non-negative matrix factorization incorporating music-specific constraints
Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on
Cited by (6)
Deep matrix factorization models for estimation of missing data in a low-cost sensor network to measure air quality
2022, Ecological InformaticsCitation Excerpt :Thus, authors have sought to generate new algorithms that can learn the representations of the characteristics from scratch (Zhang et al., 2019) and have the mathematical interpretation of MF models (De Handschutter et al., 2021; Sainath et al., 2013). As a result, MF-based models have demonstrated efficiency in signal reconstruction (Hautecoeur and Glineur, 2020) and could be used as an effective alternative to estimate PM2.5 in a low-cost WSN, i.e., the sensor network of Ciudadanos Científicos, in which approximately 40% of the information is missing or anomalous. Nevertheless, PM2.5 measurements are also affected by other variables such as time of the day (higher in high traffic hours) and localization (higher near downtown and factory areas).
Least-Squares Methods for Nonnegative Matrix Factorization Over Rational Functions
2023, IEEE Transactions on Signal ProcessingMissing Data Estimation in a Low-Cost Sensor Network for Measuring Air Quality: a Case Study in Aburrá Valley
2021, Water, Air, and Soil PollutionHierarchical alternating nonlinear least squares for nonnegative matrix factorization using rational functions
2021, European Signal Processing Conference
François Glineur received dual engineering degrees from Universit de Mons and CentraleSuplec in 1997, and a PhD in Applied Sciences from Universit de Mons in 2001. He visited Delft University of Technology and McMaster University as a post-doctoral researcher, then joined Universit catholique de Louvain where he is currently a professor of applied mathematics at the Engineering School, member of the Center for Operations Research and Econometrics and the Institute of Information and Communication Technologies, Electronics and Applied Mathematics. His research interests focus on optimization models and methods (mainly convex optimization and algorithmic efficiency) and their engineering applications, as well as on nonnegative matrix factorization and applications to data analysis.
Cécile Hautecoeur is a Ph.D. student in the ICTEAM institute of the Universit Catholique de Louvain. She received a master’s degree in applied mathematics in 2018 in the same university. Her research interests are nonnegative matrix factorization with a special focus on high dimensional or continuous data.
- ☆
This work was supported by the Fonds de la Recherche Scientifique - FNRS and the Fonds Wetenschappelijk Onderzoek - Vlaanderen under EOS Project no 30468160.
- ☆☆
An implementation of the algorithm presented in this article, as well as code generating the figures are available on Code Ocean at url: https://codeocean.com/capsule/2082198/tree.