A non-negative matrix factorization model based on the zero-inflated Tweedie distribution

Abstract

Non-negative matrix factorization (NMF) is a technique of multivariate analysis used to approximate a given matrix containing non-negative data using two non-negative factor matrices that has been applied to a number of fields. However, when a matrix containing non-negative data has many zeroes, NMF encounters an approximation difficulty. This zero-inflated situation occurs often when a data matrix is given as count data, and becomes more challenging with matrices of increasing size. To solve this problem, we propose a new NMF model for zero-inflated non-negative matrices. Our model is based on the zero-inflated Tweedie distribution. The Tweedie distribution is a generalization of the normal, the Poisson, and the gamma distributions, and differs from each of the other distributions in the degree of robustness of its estimated parameters. In this paper, we show through numerical examples that the proposed model is superior to the basic NMF model in terms of approximation of zero-inflated data. Furthermore, we show the differences between the estimated basis vectors found using the basic and the proposed NMF models for $\beta$ divergence by applying it to real purchasing data.

Appendix: Proof of (25)

${\text {Case of } \beta < 1}$

The differential of the auxiliary function (17) is given by

$$\begin{aligned} \frac{\partial Q_{\text {aux}}(\varvec{F},\varvec{A})}{\partial f_{im}} = \textstyle \sum _{j}z_{ij}^{*}\eta _{ij}^{\beta -1}a_{jm} - \sum _{j}z_{ij}^{*}y_{ij}f_{im}^{\beta -2}\lambda _{ijm}^{2-\beta }a_{jm}^{\beta -1}. \end{aligned}$$

(30)

The update equation for $f_{im}$ is derived from (30) when it is zero, as follows:

$$\begin{aligned}&\textstyle \sum _{j}z_{ij}^{*}\eta _{ij}^{\beta -1}a_{jm} - \sum _{j}z_{ij}^{*}y_{ij}f_{im}^{\beta -2}\lambda _{ijm}^{2-\beta }a_{jm}^{\beta -1} = 0 \nonumber \\ \Longleftrightarrow&f_{im}^{\beta -2}\textstyle \sum _{j}z_{ij}^{*}y_{ij}\lambda _{ijm}^{2-\beta }a_{jm}^{\beta -1} = \sum _{j}z_{ij}^{*}\eta _{ij}^{\beta -1}a_{jm} \nonumber \\ \Longleftrightarrow&f_{im}^{2-\beta } = \frac{\sum _{j}z_{ij}^{*}y_{ij}\lambda _{ijm}^{2-\beta }a_{jm}^{\beta -1}}{\sum _{j}z_{ij}^{*}\eta _{ij}^{\beta -1}a_{jm}} \nonumber \\ \Longleftrightarrow&f_{im} = \left\{ \frac{\sum _{j}z_{ij}^{*}y_{ij}\lambda _{ijm}^{2-\beta }a_{jm}^{\beta -1}}{\sum _{j}z_{ij}^{*}\eta _{ij}^{\beta -1}a_{jm}} \right\} ^{\frac{1}{2-\beta }} \end{aligned}$$

(31)

${\text {Case of } 1 \le \beta \le 2}$

The differential of the auxiliary function (17) is given by

$$\begin{aligned} \frac{\partial Q_{\text {aux}}(\varvec{F},\varvec{A})}{\partial f_{im}} = \textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }f_{im}^{\beta -1}a_{jm}^{\beta } - \sum _{j}z_{ij}^{*}y_{ij}\lambda _{ijm}^{2-\beta }f_{im}^{\beta -2}a_{jm}^{\beta -1}. \end{aligned}$$

(32)

The update equation for $f_{im}$ is derived from (32) when it is zero, as follows:

$$\begin{aligned}&\textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }f_{im}^{\beta -1}a_{jm}^{\beta } - \sum _{j}z_{ij}^{*}y_{ij}\lambda _{ijm}^{2-\beta }f_{im}^{\beta -2}a_{jm}^{\beta -1} = 0 \nonumber \\ \Longleftrightarrow&f_{im}^{\beta -1}\textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }a_{jm}^{\beta } = f_{im}^{\beta -2}\sum _{j}z_{ij}^{*}y_{ij}\lambda _{ijm}^{2-\beta }a_{jm}^{\beta -1} \nonumber \\ \Longleftrightarrow&f_{im}\textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }a_{jm}^{\beta } = \sum _{j}z_{ij}^{*}y_{ij}\lambda _{ijm}^{2-\beta }a_{jm}^{\beta -1} \nonumber \\ \Longleftrightarrow&f_{im} = \frac{\sum _{j}z_{ij}^{*}y_{ij}\lambda _{ijm}^{2-\beta }a_{jm}^{\beta -1}}{\textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }a_{jm}^{\beta }} \end{aligned}$$

(33)

${\text {Case of }\beta > 2}$

The differential of the auxiliary function (17) is given by

$$\begin{aligned} \frac{\partial Q_{\text {aux}}(\varvec{F},\varvec{A})}{\partial f_{im}} = \textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }f_{im}^{\beta -1}a_{jm}^{\beta } - \sum _{j}z_{ij}^{*}y_{ij}\eta _{ij}^{\beta -2}a_{jm}. \end{aligned}$$

(34)

The update equation for $f_{im}$ is derived from (34) when it is zero, as follows:

$$\begin{aligned}&\textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }f_{im}^{\beta -1}a_{jm}^{\beta } - \sum _{j}z_{ij}^{*}y_{ij}\eta _{ij}^{\beta -2}a_{jm} = 0 \nonumber \\ \Longleftrightarrow&f_{im}^{\beta -1}\textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }a_{jm}^{\beta } = \sum _{j}z_{ij}^{*}y_{ij}\eta _{ij}^{\beta -2}a_{jm} \nonumber \\ \Longleftrightarrow&f_{im}^{\beta -1} = \frac{\sum _{j}z_{ij}^{*}y_{ij}\eta _{ij}^{\beta -2}a_{jm}}{\textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }a_{jm}^{\beta }} \nonumber \\ \Longleftrightarrow&f_{im} = \left\{ \frac{\sum _{j}z_{ij}^{*}y_{ij}\eta _{ij}^{\beta -2}a_{jm}}{\textstyle \sum _{j}z_{ij}^{*}\lambda _{ijm}^{1-\beta }a_{jm}^{\beta }}\right\} ^{\frac{1}{\beta -1}} \end{aligned}$$

(35)

We obtain (25) replacing $f_{im}$, $a_{jm}$, $\lambda _{ijm}$, and $\eta _{ij}$ to $f_{im}^{(t)}$, $a_{jm}^{(t-1)}$, $\lambda _{ijm}^{(f)}$, and $\eta _{ij}^{(f)}$, respectively in (30), (32), and (34). $\blacksquare$