A deep discriminative and robust nonnegative matrix factorization network method with soft label constraint

Abstract

In order to obtain a discriminative, compact and robust data representation, a discriminative and robust nonnegative matrix factorization method with soft label constraint (DRNMF_SLC) is proposed. By minimizing the objective function, the data representation after learning soft label constraint is obtained. To further acquire a more hierarchical and discriminative data representation, a deep discriminative and robust nonnegative matrix factorization network method with soft label constraint (Deep DRNMFN_SLC) is constructed. In order to improve the feature expression ability of deep neural network (DNN), a deep discriminative and robust nonnegative matrix factorization network method with soft label constraint based on DNN (Deep DRNMFN_SLC_DNN) is proposed, which could obtain a more discriminative, robust and generalized feature representation, and meanwhile greatly reduce the dimension of data features. Furthermore, the objective function of DRNMF_SLC is constructed by introducing both the global loss function and the central loss function of soft label constraint matrix, and the optimization solution and convergence proof of objective function are given simultaneously. When the proposed DRNMF_SLC method and Deep DRNMFN_SLC_DNN method are, respectively, applied to the face recognition under occlusions and illumination variations, the frameworks, Algorithm 1 and Algorithm 2 are given. The extensive and adequate experiments demonstrate the effectiveness of the proposed method.

Appendix: Proof of Theorem 1

To prove Theorem 1, the following characteristic of an auxiliary function is used, which is the same as used in the expectation–maximization (EM) algorithm.

Lemma 1

If there is an auxiliary function $G$ for $\tilde{J}\left( x \right)$, which satisfies the conditions of $G\left( {x,x{\kern 1pt}^{t} } \right) \ge \tilde{J}\left( x \right)$ and $G\left( {x,x} \right)\text{ = }\tilde{J}\left( x \right)$, then $\tilde{J}\left( x \right)$ is non-increasing under the update

$$ x{\kern 1pt}^{t + 1} \text{ = }\arg \min_{x} G\left( {x,x{\kern 1pt}^{t} } \right) $$

(42)

The equality $\tilde{J}\left( {x^{{{\kern 1pt} t + 1}} } \right)\text{ = }\tilde{J}\left( {x^{{{\kern 1pt} t}} } \right)$ holds only if $x{\kern 1pt}^{t}$ is a local minimum of $G\left( {x,x^{{{\kern 1pt} t}} } \right)$. By iterating the update in Eq. (42), a sequence of estimates that converge to a local minimum $x_{\hbox{min} } \text{ = }\arg \min_{x} \tilde{J}\left( x \right)$ is obtained, which will be shown by defining a proper auxiliary function for the objective function in Eq. (13).

First, the convergence of update rule in Eq. (21) is proven. For any an element $A_{j,q} \left( k \right)$ in ${\mathbf{A}}\left( k \right)$, let $\tilde{J}_{{A_{j,q} \left( k \right)}}$ denote the part of $J_{DRNMF\_SLC}$ relevant to $A_{j,q} \left( k \right)$. Since the update is element wise essentially, it is sufficient to reveal that each $\tilde{J}_{{A_{j,q} \left( k \right)}}$ is non-increasing under the update rule of Eq. (21), which is proven by defining the auxiliary function with regard to $A_{j,q} \left( k \right)$ as follows:

Lemma 2

Let $\tilde{J}^{\prime}$ represent the first-order derivative with regard to ${\mathbf{A}}\left( k \right)$. The function

$$ \begin{aligned} G\left( {A\left( k \right),A_{j,q}^{t} \left( k \right)} \right)\text{ = }\tilde{J}_{{A_{j,q} \left( k \right)}} \left( {A_{j,q}^{t} \left( k \right)} \right)\text{ + }\tilde{J}^{\prime}_{{A_{j,q} \left( k \right)}} \left( {A_{j,q}^{t} \left( k \right)} \right)\left( {A\left( k \right) - A_{j,q}^{t} \left( k \right)} \right) \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \text{ + }\frac{{\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{j,q} }}{{A_{j,q}^{t} \left( k \right)}}\left( {A\left( k \right) - A_{j,q}^{t} \left( k \right)} \right)^{2} \\ \end{aligned} $$

(43)

is an auxiliary function for $\tilde{J}_{{A_{j,q} \left( k \right)}}$, which is the part of $J_{DRNMF\_SLC}$ that is only related to $A_{j,q} \left( k \right)$.

Proof

Obviously, $G\left( {A\left( k \right),A\left( k \right)} \right)\text{ = }\tilde{J}_{{A_{j,q} \left( k \right)}} \left( {A\left( k \right)} \right)$. Based on the definition of auxiliary function, only $G\left( {A\left( k \right),A_{j,q}^{t} \left( k \right)} \right) \ge \tilde{J}_{{A_{j,q} \left( k \right)}} \left( {A\left( k \right)} \right)$ needs to be demonstrated. In order to achieve this purpose, $G\left( {A\left( k \right),A_{j,q}^{t} \left( k \right)} \right)$ in Eq. (43) is compared with the Taylor series expansion of $\tilde{J}_{{A_{j,q} \left( k \right)}} \left( {A\left( k \right)} \right)$:

$$ \tilde{J}_{{A_{j,q} \left( k \right)}} \left( {A\left( k \right)} \right)\text{ = }\tilde{J}_{{A_{j,q} \left( k \right)}} \left( {A_{j,q}^{t} \left( k \right)} \right)\text{ + }\tilde{J}^{\prime}_{{A_{j,q} \left( k \right)}} \left( {A\left( k \right) - A_{j,q}^{t} \left( k \right)} \right)\text{ + }\frac{1}{2}\tilde{J}^{\prime\prime}_{{A_{j,q} \left( k \right)}} \left( {A\left( k \right) - A_{j,q}^{t} \left( k \right)} \right)^{2} $$

(44)

where $\tilde{J}^{\prime\prime}$ is the second-order derivative regarding ${\mathbf{A}}\left( k \right)$. It is simple to check that

$$ \tilde{J}^{\prime}_{{A_{j,q} \left( k \right)}} \text{ = }\left( {\frac{{\partial J_{DRNMF\_SLC} }}{{\partial {\mathbf{A}}\left( k \right)}}} \right)_{j,q} \text{ = }\left( { - 2{\mathbf{Z}}^{T} \left( k \right){\mathbf{B}}\left( k \right){\mathbf{F}}^{T} \left( k \right)\text{ + }2{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right){\mathbf{F}}^{{{\kern 1pt} T}} \left( k \right)} \right)_{j,q} $$

(45)

$$ \tilde{J}^{\prime\prime}_{{A_{j,q} \left( k \right)}} \text{ = }2\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right)} \right)_{j,j} \left( {{\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{q,q} $$

(46)

Substituting Eq. (46) into Eq. (44) and comparing with Eq. (43), it can be seen that, instead of demonstrating $G\left( {A\left( k \right),A_{j,q}^{t} \left( k \right)} \right) \ge \tilde{J}_{{A_{j,q} \left( k \right)}} \left( {A\left( k \right)} \right)$, it is equivalent to prove

$$ \frac{{\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{j,q} }}{{A_{j,q}^{t} \left( k \right)}} \ge \frac{1}{2}\tilde{J}^{\prime\prime}_{{A_{j,q} \left( k \right)}} \text{ = }\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right)} \right)_{j,j} \left( {{\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{q,q} $$

(47)

To prove the above inequality, the following inequality holds as:

$$ \begin{aligned} \left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{j,q} \text{ = }\sum\limits_{l} {\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right)} \right)_{j,l} \left( {{\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{l,q} } \\ \ge \left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right)} \right)_{j,q} \left( {{\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{q,q} \\ \ge \sum\limits_{l} {\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right)} \right)_{j,l} A_{l,q}^{t} \left( k \right)\left( {{\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{q,q} } \\ \ge A_{j,q}^{t} \left( k \right)\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right)} \right)_{j,j} \left( {{\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{q,q} \\ \end{aligned} $$

(48)

In summary, the statement $G\left( {A\left( k \right),A_{j,q}^{t} \left( k \right)} \right) \ge \tilde{J}_{{A_{j,q} \left( k \right)}} \left( {A\left( k \right)} \right)$ holds, and Lemma 2 is proved.

Subsequently, an auxiliary function is defined for the update rule in Eq. (18). Similarly, let $\tilde{J}_{{Z_{i,j} \left( k \right)}}$ represent the part of $J_{DRNMF\_SLC}$ related to $Z_{i,j} \left( k \right)$. Then, the auxiliary function relevant to $Z_{i,j} \left( k \right)$ is defined as follows:

Lemma 3

The function

$$ \begin{aligned} G\left( {Z\left( k \right),Z_{i,j}^{t} \left( k \right)} \right)\text{ = }\tilde{J}_{{Z_{i,j} \left( k \right)}} \left( {Z_{i,j}^{t} \left( k \right)} \right)\text{ + }\tilde{J}^{\prime}_{{Z_{i,j} \left( k \right)}} \left( {Z_{i,j}^{t} \left( k \right)} \right)\left( {Z\left( k \right) - Z_{i,j}^{t} \left( k \right)} \right) \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \text{ + }\frac{{\left( {{\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right){\mathbf{A}}^{T} \left( k \right)} \right)_{i,j} }}{{Z_{i,j}^{t} \left( k \right)}}\left( {Z\left( k \right) - Z_{i,j}^{t} \left( k \right)} \right)^{2} \\ \end{aligned} $$

(49)

is an auxiliary function for $\tilde{J}_{{Z_{i,j} \left( k \right)}}$, which is the part of $J_{DRNMF\_SLC}$ that is only related to $Z_{i,j} \left( k \right)$.

The proof of Lemma 3 is similar to the proof of Lemma 2 essentially and is omitted due to space limitation.

Lemma 4

The function

$$ \begin{aligned} G\left( {F\left( k \right),F_{q,a}^{t} \left( k \right)} \right)\text{ = }\tilde{J}_{{F_{q,a} \left( k \right)}} \left( {F_{q,a}^{t} \left( k \right)} \right)\text{ + }\tilde{J}^{\prime}_{{F_{q,a} \left( k \right)}} \left( {F_{q,a}^{t} \left( k \right)} \right)\left( {F\left( k \right) - F_{q,a}^{t} \left( k \right)} \right) \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\text{ + }}\frac{{\left( {{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} {\text{ + }}\left( {{\mathbf{F}}\left( k \right)} \right)_{q,a} + \lambda_{{{\kern 1pt} 1}} \left( {\eta_{p,q}^{\left( r \right)} \left( k \right)} \right){\text{ + }}\frac{{\lambda_{3} }}{2}\left( {{\mathbf{D}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} }}{{F_{q,a}^{t} \left( k \right)}}\left( {F\left( k \right) - F_{q,a}^{t} \left( k \right)} \right)^{2} \\ \end{aligned} $$

(50)

is an auxiliary function for $\tilde{J}_{{F_{q,a} \left( k \right)}}$, which is the part of $J_{DRNMF\_SLC}$ that is only related to $F_{q,a} \left( k \right)$.

Proof

Obviously, $G\left( {F\left( k \right),F\left( k \right)} \right)\text{ = }\tilde{J}_{{F_{q,a} \left( k \right)}} \left( {F\left( k \right)} \right)$. Based on the definition of auxiliary function, only $G\left( {F\left( k \right),F_{q,a}^{{{\kern 1pt} t}} \left( k \right)} \right) \ge \tilde{J}_{{F_{q,a} \left( k \right)}} \left( {F\left( k \right)} \right)$ needs to be proven. In order achieve this goal, $G\left( {F\left( k \right),F_{q,a}^{{{\kern 1pt} t}} \left( k \right)} \right)$ in Eq. (50) is compared with the Taylor series expansion of $\tilde{J}_{{F_{q,a} \left( k \right)}} \left( {F\left( k \right)} \right)$:

$$ \tilde{J}_{{F_{q,a} \left( k \right)}} \left( {F\left( k \right)} \right)\text{ = }\tilde{J}_{{F_{q,a} \left( k \right)}} \left( {F_{q,a}^{t} \left( k \right)} \right)\text{ + }\tilde{J}^{\prime}_{{F_{q,a} \left( k \right)}} \left( {F\left( k \right) - F_{q,a}^{t} \left( k \right)} \right)\text{ + }\frac{1}{2}\tilde{J}^{\prime\prime}_{{F_{q,a} \left( k \right)}} \left( {F\left( k \right) - F_{q,a}^{t} \left( k \right)} \right)^{2} $$

(51)

where

$$ \begin{aligned} \tilde{J}^{\prime}_{{F_{q,a} \left( k \right)}} \text{ = }\left( { - 2{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{B}}\left( k \right)\text{ + }2{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} \text{ + }2\left( {F_{q,a} \left( k \right) - C_{q,a} \left( k \right)} \right) \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \text{ + }2\lambda_{{{\kern 1pt} 1}} \left( {\eta_{p,q}^{\left( r \right)} \left( k \right) - \mu_{q}^{\left( r \right)} \left( k \right)} \right) - \frac{{4\lambda_{2} }}{{N_{r} }}\sum\limits_{i \ne r}^{c} {\left( {\mu_{q}^{\left( r \right)} \left( k \right) - \mu_{q}^{\left( i \right)} \left( k \right)} \right)} \text{ + }\lambda_{3} \left( {{\mathbf{D}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} \\ \end{aligned} $$

(52)

$$ \tilde{J}^{\prime\prime}_{{F_{q,a} \left( k \right)}} \text{ = }2\left( {{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right)} \right)_{q,q} \text{ + }2\text{ + }2\lambda_{{{\kern 1pt} 1}} - \frac{{2\lambda_{1} }}{{N_{r} }} - \frac{{4\lambda_{2} \left( {c - 1} \right)}}{{N_{r}^{2} }}\text{ + }\lambda_{3} \left( {{\mathbf{D}}\left( k \right)} \right)_{q,q} $$

(53)

Instead of demonstrating $G\left( {F\left( k \right),F_{q,a}^{t} \left( k \right)} \right) \ge \tilde{J}_{{F_{q,a} \left( k \right)}} \left( {F\left( k \right)} \right)$, it is equivalent to prove

$$ \frac{{\left( {{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} {\text{ + }}\left( {{\mathbf{F}}\left( k \right)} \right)_{q,a} {\text{ + }}\lambda_{{{\kern 1pt} 1}} \left( {\eta_{p,q}^{\left( r \right)} \left( k \right)} \right) + \frac{{\lambda_{3} }}{2}\left( {{\mathbf{D}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} }}{{F_{q,a}^{t} \left( k \right)}} \ge \frac{1}{2}\tilde{J}^{\prime\prime}_{{F_{q,a} \left( k \right)}} $$

(54)

To prove above inequality, we have

$$ \begin{aligned} \left( {{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} \text{ = }\sum\limits_{l} {\left( {{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right)} \right)_{q,l} } \left( {{\mathbf{F}}\left( k \right)} \right)_{l,a} \\ \ge \left( {{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right)} \right)_{q,q} \left( {{\mathbf{F}}\left( k \right)} \right)_{q,a} \\ \end{aligned} $$

(55)

$$ \begin{aligned} \left( {{\mathbf{D}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} \text{ = }\sum\limits_{l} {\left( {{\mathbf{D}}\left( k \right)} \right)_{q,l} } \left( {{\mathbf{F}}\left( k \right)} \right)_{l,a} \\ \ge \left( {{\mathbf{D}}\left( k \right)} \right)_{q,q} \left( {{\mathbf{F}}\left( k \right)} \right)_{q,a} \\ \end{aligned} $$

(56)

Thus, the following inequality holds:

$$ \frac{{\left( {{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} {\text{ + }}\left( {{\mathbf{F}}\left( k \right)} \right)_{q,a} {\text{ + }}\lambda_{{{\kern 1pt} 1}} \left( {\eta_{p,q}^{\left( r \right)} \left( k \right)} \right){\text{ + }}\frac{{\lambda_{3} }}{2}\left( {{\mathbf{D}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} }}{{F_{q,a}^{t} \left( k \right)}} \ge \frac{1}{2}\tilde{J}^{\prime\prime}_{{F_{q,a} \left( k \right)}} $$

(57)

As a result, the statement $G\left( {F\left( k \right),F_{q,a}^{t} \left( k \right)} \right) \ge \tilde{J}_{{F_{q,a} \left( k \right)}} \left( {F\left( k \right)} \right)$ holds, and Lemma 4 is proved.

With the above lemmas, the proof of Theorem 1 is further given.

Proof of Theorem 1

Substituting $G\left( {A\left( k \right),A_{j,q}^{t} \left( k \right)} \right)$ of Eq. (43) into Eq. (42), the following equation is obtained:

$$ A_{j,q}^{t + 1} \left( k \right) = \text{arg}\min_{A\left( k \right)} G\left( {A\left( k \right),A_{j,q}^{t} \left( k \right)} \right)\text{ = }A_{j,q}^{t} \left( k \right)\frac{{\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{B}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{j,q} }}{{\left( {{\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right)} \right)_{j,q} }} $$

(58)

Since Eq. (43) is an auxiliary function, $\tilde{J}_{{A_{j,q} \left( k \right)}}$ is non-increasing under this iteration rule in Eq. (58), according to Lemma 2.

Then, substituting $G\left( {Z\left( k \right),Z_{i,j}^{t} \left( k \right)} \right)$ of Eq. (49) into Eq. (42), the following equation is obtained:

$$ Z_{i,j}^{t + 1} \left( k \right)\text{ = arg}\min_{Z\left( k \right)} G\left( {Z\left( k \right),Z_{i,j}^{t} \left( k \right)} \right)\text{ = }Z_{i,j}^{t} \left( k \right)\frac{{\left( {{\mathbf{B}}\left( k \right){\mathbf{F}}^{T} \left( k \right){\mathbf{A}}^{T} \left( k \right)} \right)_{i,j} }}{{\left( {{\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right){\mathbf{F}}^{T} \left( k \right){\mathbf{A}}^{T} \left( k \right)} \right)_{i,j} }} $$

(59)

Since Eq. (49) is an auxiliary function, $\tilde{J}_{{Z_{i,j} \left( k \right)}}$ is non-increasing under this iteration rule in Eq. (59), according to Lemma 3.

Similarly, substituting $G\left( {F\left( k \right),F_{q,a}^{t} \left( k \right)} \right)$ of Eq. (50) into Eq. (42), the following equation is obtained:

$$ \begin{aligned} F_{q,a}^{t + 1} \left( k \right)\text{ = arg}\min_{F\left( k \right)} G\left( {F\left( k \right),F_{q,a}^{t} \left( k \right)} \right) \\ \text{ = }F_{q,a}^{t} \left( k \right)\frac{{\left( {2{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{B}}\left( k \right)} \right)_{q,a} {\text{ + }}\frac{{4\lambda_{2} }}{{N_{r} }}\sum\limits_{i \ne r}^{c} {\left( {\mu_{q}^{\left( r \right)} \left( k \right) - \mu_{q}^{\left( i \right)} \left( k \right)} \right)} }}{{\left( {2{\mathbf{A}}^{T} \left( k \right){\mathbf{Z}}^{T} \left( k \right){\mathbf{Z}}\left( k \right){\mathbf{A}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} {\text{ + }}2\left( {F_{q,a} \left( k \right) - C_{q,a} \left( k \right)} \right){\text{ + }}2\lambda_{{{\kern 1pt} 1}} \left( {\eta_{p,q}^{\left( r \right)} \left( k \right) - \mu_{q}^{\left( r \right)} \left( k \right)} \right){\text{ + }}\lambda_{3} \left( {{\mathbf{D}}\left( k \right){\mathbf{F}}\left( k \right)} \right)_{q,a} }} \\ \end{aligned} $$

(60)

Since Eq. (50) is an auxiliary function, $\tilde{J}_{{F_{q,a} \left( k \right)}}$ is non-increasing under this iteration rule in Eq. (60) according to Lemma 4.

According to the above derivation and proof, the update rules of ${\mathbf{Z}}\left( k \right)$, ${\mathbf{A}}\left( k \right)$ and ${\mathbf{F}}\left( k \right)$ in Eqs. (18), (21) and (32) result in a series of non-increasing values of $J_{DRNMF\_SLC}$ and hence a local minimum.