An explainable multi-sparsity multi-kernel nonconvex optimization least-squares classifier method via ADMM

Abstract

Convex optimization techniques are extensively applied to various models, algorithms, and applications of machine learning and data mining. For optimization-based classification methods, the sparsity principle can greatly help to select simple classifier models, while the single- and multi-kernel methods can effectively address nonlinearly separable problems. However, the limited sparsity and kernel methods hinder the improvement of the predictive accuracy, efficiency, iterative update, and interpretable classification model. In this paper, we propose a new Explainable Multi-sparsity Multi-kernel Nonconvex Optimization Least-squares Classifier (EM²NOLC) model, which is an optimization problem with a least-squares objective function and multi-sparsity multi-kernel nonconvex constraints, aiming to address the aforementioned issues. Based on reconstructed multiple kernel learning (MKL), the proposed model can extract important instances and features by finding the sparse coefficient and kernel weight vectors, which are used to compute importance or contribution to classification and obtain the explainable prediction. The corresponding EM²NOLC algorithm is implemented with the Alternating Direction Method of Multipliers (ADMM) method. On the real classification datasets, compared with the three ADMM classifiers of Linear Support Vector Machine Classifier, SVMC, Least Absolute Shrinkage and Selection Operator Classifier, the two MKL classifiers of SimpleMKL and EasyMKL, and the gradient descent classifier of Feature Selection for SVMC, our proposed EM²NOLC generally obtains the best predictive performance and explainable results with the least number of important instances and features having different contribution percentages.

Appendices

Appendix

1.1 Proof of equality (6)

For any two input points ${\varvec{x}}_{i}$ and ${\varvec{x}}_{j}$ ($i,j = 1, \cdots ,n$) from the training set ${\varvec{T}}$, suppose that a basis function $\phi ( \cdot )$ mapping any feature value ${\varvec{x}}_{jm}$($m = 1, \cdots ,d$) from the input space to a new feature space is given. If their product $\phi ({\varvec{x}}_{jm} ) \times \phi ({\varvec{x}}_{im} )$ in the feature space can be replaced with the kernel function $\kappa (x_{jm} , x_{im} )$ regarding the $m{\text{th}}$ feature. The kernel vector ${\varvec{u}}_{ji}$(${\varvec{u}}_{ji} \in {\mathbb{R}}^{d}$) of $d$ features is denoted as

$$\begin{gathered} {\varvec{u}}_{ji} = \left[ {\phi (x_{j1} ), \cdots ,\phi (x_{jd} )} \right] \odot \left[ {\phi (x_{i1} ), \cdots ,\phi (x_{id} )} \right] \hfill \\ \, \quad \quad = \left[ {\phi (x_{j1} )\phi (x_{i1} ), \cdots ,\phi (x_{jd} )\phi (x_{id} )} \right] \hfill \\ \, \quad \quad= \left[ {\kappa (x_{j1} ,x_{i1} ), \cdots ,\kappa (x_{jd} ,x_{id} )} \right]. \hfill \\ \end{gathered}$$

Given the kernel weight vector ${\varvec{\mu}}^{t}$ at iteration $t$, the row-wise multi-kernel vector ${\varvec{A}}_{j}$(${\varvec{A}}_{j} \in {\mathbb{R}}^{n}$) is defined as a weighted similarity between the input points ${\varvec{x}}_{j}$ and other input points in the training set, that is we have the equality

$${\varvec{A}}_{j} = \left( {{\varvec{u}}_{j1} {\varvec{\mu}}^{t} , \cdots ,{\varvec{u}}_{jn} {\varvec{\mu}}^{t} } \right)^{T} ,\;\;j = 1, \cdots ,n$$

The row-wise multi-kernel matrix ${\varvec{A}}$(${\varvec{A}} \in {\mathbb{R}}^{n \times n}$) has the below form

$${\varvec{A}} = \left( {{\varvec{A}}_{1} , \cdots ,{\varvec{A}}_{n} } \right)$$

So, for any two input points ${\varvec{x}}_{i}$ and ${\varvec{x}}_{j}$, the element of the matrix ${\varvec{A}}$ is ${\varvec{A}}_{ji} = \sum\nolimits_{m = 1}^{d} {\mu_{m}^{t} \kappa ({\varvec{x}}_{jm} , \, {\varvec{x}}_{im} )}$ for all $i,j = 1, \cdots ,n$.

1.2 Proof of equality (8)

For any feature ${\varvec{f}}_{m}$(${\varvec{f}}_{m} \in {\mathbb{R}}^{n}$, $m = 1, \cdots ,d$) from the training set ${\varvec{T}}$, assume that the mapping function $\phi ( \cdot )$ transforms the feature value ${\varvec{x}}_{jm}$($j = 1, \cdots ,n$) in the input space into a new feature space, the Kronecker product ${\varvec{P}}$ (${\varvec{P}} \in {\mathbb{R}}^{n \times n}$) regarding feature ${\varvec{f}}_{m}$ is computed by

$$\begin{gathered} {\varvec{P}} = {\varvec{f}}_{m} {\varvec{f}}_{m}^{T} \hfill \\ = \left[ {\phi (x_{1m} ), \cdots ,\phi (x_{nm} )} \right]^{T} \left[ {\phi (x_{1m} ), \cdots ,\phi (x_{nm} )} \right] \hfill \\ \, = \left( {\begin{array}{*{20}c} {\phi (x_{1m} )\phi (x_{1m} )} & \ldots & {\phi (x_{1m} )\phi (x_{nm} )} \\ \vdots & \ddots & \vdots \\ {\phi (x_{nm} )\phi (x_{1m} )} & \cdots & {\phi (x_{nm} )\phi (x_{nm} )} \\ \end{array} } \right) \hfill \\ \end{gathered}$$

$$= \left( {\begin{array}{*{20}c} {\kappa (x_{1m} ,x_{1m} )} & \ldots & {\kappa (x_{1m} ,x_{nm} )} \\ \vdots & \ddots & \vdots \\ {\kappa (x_{nm} ,x_{1m} )} & \cdots & {\kappa (x_{nm} ,x_{nm} )} \\ \end{array} } \right).$$

If the coefficient vector ${\varvec{\lambda}}^{t}$ at iteration $t$ is given, then the column-wise multi-kernel vector ${\varvec{B}}_{m}$(${\varvec{B}}_{m} \in {\mathbb{R}}^{n}$) with respect to the $m{\text{th}}$ feature can be obtained by the multiplication of the transposed matrix ${\varvec{P}}^{T}$ and the vector ${\varvec{\lambda}}^{t}$ with the form

$$\begin{gathered} {\varvec{B}}_{m} = {\varvec{P}}^{T} {\varvec{\lambda}}^{t} \hfill \\ { = }\left( {\begin{array}{*{20}c} {\kappa (x_{1m} ,x_{1m} )} & \ldots & {\kappa (x_{1m} ,x_{nm} )} \\ \vdots & \ddots & \vdots \\ {\kappa (x_{nm} ,x_{1m} )} & \cdots & {\kappa (x_{nm} ,x_{nm} )} \\ \end{array} } \right)^{T} \left[ \begin{gathered} \lambda_{1}^{t} \hfill \\ \vdots \hfill \\ \lambda_{n}^{t} \hfill \\ \end{gathered} \right] \hfill \\ \, = \left[ {\sum\limits_{j = 1}^{n} {\lambda_{j}^{t} \kappa (x_{jm} ,x_{1m} )} , \cdots ,\sum\limits_{j = 1}^{n} {\lambda_{j}^{t} \kappa (x_{jm} ,x_{nm} )} } \right]^{T} . \hfill \\ \end{gathered}$$

The column-wise multi-kernel matrix ${\varvec{B}}$(${\varvec{B}} \in {\mathbb{R}}^{n \times d}$) is denoted as.

$\user2{B = }\left( {{\varvec{B}}_{1} , \cdots ,{\varvec{B}}_{d} } \right)$.

Thus, for any input points ${\varvec{x}}_{i}$ with respect to the feature ${\varvec{f}}_{m}$, the element of the matrix ${\varvec{B}}$ is ${\varvec{B}}_{im} = \sum\nolimits_{j = 1}^{n} {\lambda_{j}^{t} \kappa ({\varvec{x}}_{jm} , \, {\varvec{x}}_{im} )}$ for all $i = 1, \cdots ,n$ and $m = 1, \cdots ,d$.

1.3 Proof of the ${\mathbf{\lambda - step}}$ EM²NOLC algorithm via ADMM

Corresponding with the $\lambda {\text{ - step}}$ EM²NOLC model (7) and its ADMM optimization problem (13) with the separable objective and equality constraint functions, the augmented Lagrangian function regarding the scaled dual variables ${\varvec{u}}$(${\varvec{u}} \in {\mathbb{R}}^{n}$) and the penalty parameter $\rho$($\rho > 0$) is defined as.

${\varvec{L}}_{\rho } ({\varvec{\lambda}}, \, {\varvec{q}}, \, {\varvec{u}}) = f({\varvec{\lambda}}) + g({\varvec{q}}) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\lambda}} - \, {\varvec{q}} + {\varvec{u}}} \right\|_{2}^{2} - \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {\varvec{u}} \right\|_{2}^{2}$

The ADMM updates (17), (18), and (16) of the $\lambda {\text{ - step}}$ EM²NOLC algorithm are obtained from the partial derivative of ${\varvec{L}}_{\rho } (\lambda , \, {\varvec{q}}, \, {\varvec{u}})$ with respect to its three parameters ${\varvec{\lambda}}$, ${\varvec{q}}$, and ${\varvec{u}}$, respectively. We can express the $\lambda {\text{ - minimization}}$ as the proximal operator:

$$\begin{gathered} {\varvec{\lambda}}^{k + 1} = \mathop {\arg \min }\limits_{{\lambda \in {\mathbb{R}}^{n} }} {\varvec{L}}_{\rho } ({\varvec{\lambda}}, \, {\varvec{q}}^{k} , \, {\varvec{u}}^{k} ) \hfill \\ { } = \mathop {\arg \min }\limits_{{\lambda \in {\mathbb{R}}^{n} }} \left\{ {f({\varvec{\lambda}}) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\lambda}} - {\varvec{q}}^{k} + {\varvec{u}}^{k} } \right\|_{2}^{2} } \right\}. \hfill \\ \end{gathered}$$

Then the gradient of ${\varvec{L}}_{\rho } ({\varvec{\lambda}}, \, {\varvec{q}}^{k} , \, {\varvec{u}}^{k} )$ regarding ${\varvec{\lambda}}$ is set to zero, we analytically obtained the resulting $\lambda {\text{ - update}}$:

$$\begin{gathered} \, \nabla_{\lambda } {\varvec{L}}_{\rho } ({\varvec{\lambda}}, \, {\varvec{q}}^{k} , \, {\varvec{u}}^{k} ) = 0 \hfill \\ \Rightarrow \nabla_{\lambda } \left\{ {f({\varvec{\lambda}}) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\lambda}} - {\varvec{q}}^{k} + {\varvec{u}}^{k} } \right\|_{2}^{2} } \right\} = 0 \hfill \\ \Rightarrow \nabla_{\lambda } \left\{ {({1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-\nulldelimiterspace} 2})\left\| {{\varvec{y}} \odot \left( {\user2{A\lambda } - b_{1} {\mathbf{1}}_{n} } \right) - 1_{n} } \right\|_{2}^{2} + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\lambda}} - {\varvec{q}}^{k} + {\varvec{u}}^{k} } \right\|_{2}^{2} } \right\} = 0 \hfill \\ \Rightarrow {\varvec{A}}_{y}^{T} {\varvec{A}}_{y} {\varvec{\lambda}} - {\varvec{A}}_{y}^{T} \left( {b_{1} {\varvec{y}} + {\mathbf{1}}_{n} } \right) + \rho \lambda - \rho \left( {{\varvec{q}}^{k} - {\varvec{u}}^{k} } \right) = 0 \hfill \\ \Rightarrow \lambda^{k + 1} = \left( {{\varvec{A}}_{y}^{T} {\varvec{A}}_{y} + \rho I_{n} } \right)^{ - 1} \left\{ {{\varvec{A}}_{y}^{T} \left( {b_{1} y + {\mathbf{1}}_{n} } \right) + \rho \left( {{\varvec{q}}^{k} - {\varvec{u}}^{k} } \right)} \right\}.{ } \hfill \\ \end{gathered}$$

The $q{\text{ - minimization}}$ of the $\lambda {\text{ - step}}$ EM²NOLC algorithm has the form

$$\begin{gathered} {\varvec{q}}^{k + 1} = \mathop {\arg \min }\limits_{{{\varvec{q}} \in {\mathbb{R}}^{n} }} {\varvec{L}}_{\rho } (\lambda^{k + 1} , \, {\varvec{q}}, \, {\varvec{u}}^{k} ) \hfill \\ { } = \mathop {\arg \min }\limits_{{{\varvec{q}} \in {\mathbb{R}}^{n} }} \left\{ {g({\varvec{q}}) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {\lambda^{k + 1} - {\varvec{q}} + {\varvec{u}}^{k} } \right\|_{2}^{2} } \right\}. \hfill \\ \end{gathered}$$

Dual to $S_{0} (C_{\lambda } ) = \{ {\varvec{q}} \in {\mathbb{R}}^{n} |\left\| {\varvec{q}} \right\|_{0} \le C_{\lambda } \}$ is a nonconvex set, the $q{\text{ - minimization}}$ may not converge to an optimal point. We can apply the projected gradient method to approximate the $q{\text{ - update}}$ procedure. So, we have the $q{\text{ - update}}$:

$$\begin{gathered} {\varvec{q}}^{k + 1} = \mathop {\arg \min }\limits_{{{\varvec{q}} \in {\mathbb{R}}^{n} }} \left\{ {I_{{S_{0} (C_{\lambda } )}} ({\varvec{q}}) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\lambda}}^{k + 1} - {\varvec{q}} + {\varvec{u}}^{k} } \right\|_{2}^{2} } \right\} \hfill \\ \, \approx \mathop {\arg \min }\limits_{{{\varvec{q}} \in S_{0} (C_{\lambda } )}} \left\| {{\varvec{\lambda}}^{k + 1} + {\varvec{u}}^{k} - {\varvec{q}}} \right\|_{2}^{2} \hfill \\ \, = \Pi_{{S_{0} (C_{\lambda } )}} ({\varvec{\lambda}}^{k + 1} + {\varvec{u}}^{k} ), \hfill \\ \end{gathered}$$

where the indicator function has $I_{{S_{0} (C_{\lambda } )}} ({\varvec{q}}) = 1$ if ${\varvec{q}} \in S_{0} (C_{\lambda } )$, otherwise $I_{{S_{0} (C_{\lambda } )}} ({\varvec{q}}) = 0$. For any vector ${\varvec{q}}$(${\varvec{q}} \in {\mathbb{R}}^{n}$), the projection operator $\Pi_{{S_{0} (C_{\lambda } )}} ({\varvec{q}})$ can be actually implemented by sorting the elements of the vector ${\varvec{q}}$ in descending order of their absolute values and setting all elements to zeros except top $C_{\lambda }$.

The $u{\text{ - update}}$ step can be considered as the change of constraint residuals in the optimization problem (13), which ensures the convergence of the ADMM iterations (17), (18), and (16).

1.4 Proof of the $\mu - step$ EM² NOLC algorithm via ADMM

Similar to “Proof of the λ-step EM²NOLC algorithm via ADMM” in Appendix, for the $\mu {\text{ - step}}$ EM²NOLC model (9) and its ADMM optimization problem (20) with the separable objective functions and equality constraints, the augmented Lagrangian function is denoted as.

${\varvec{L}}_{\rho } ({\varvec{\mu}}, \, {\varvec{z}}, \, {\varvec{v}}) = h({\varvec{\mu}}) + l(z) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\mu}} - \, {\varvec{z}} + {\varvec{v}}} \right\|_{2}^{2} - \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {\varvec{v}} \right\|_{2}^{2},$

with the scaled dual variables ${\varvec{v}}$(${\varvec{v}} \in {\mathbb{R}}^{d}$).

The ADMM updates (24), (25), and (23) of the $\mu {\text{ - step}}$ EM²NOLC algorithm are, respectively, generated from the KKT optimality conditions of ${\varvec{L}}_{\rho } ({\varvec{\mu}}, \, {\varvec{z}}, \, {\varvec{v}})$ regarding its three parameters ${\varvec{\mu}}$, ${\mathbf{z}}$, and ${\varvec{v}}$. The $\mu {\text{ - minimization}}$ is defined as the proximal operator:

$$\begin{gathered} {\varvec{\mu}}^{k + 1} = \mathop {\arg \min }\limits_{{\mu \in {\mathbb{R}}^{d} }} {\varvec{L}}_{\rho } ({\varvec{\mu}}, \, {\varvec{z}}^{k} , \, {\varvec{v}}^{k} ) \hfill \\ { } = \mathop {\arg \min }\limits_{{\mu \in {\mathbb{R}}^{d} }} \left\{ {h(\mu ) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\mu}} - \, {\varvec{z}}^{k} + {\varvec{v}}^{k} } \right\|_{2}^{2} } \right\}. \hfill \\ \end{gathered}$$

Setting the gradient of ${\varvec{L}}_{\rho } ({\varvec{\mu}}, \, {\varvec{z}}^{k} , \, {\varvec{v}}^{k} )$ with respect to ${\varvec{\mu}}$ to zero, we can get the analytical solution, that is the $\mu {\text{ - update}}$:

$$\begin{gathered} \, \nabla_{{\varvec{\mu}}} {\varvec{L}}_{\rho } {(}{\varvec{\mu}}, \, {\mathbf{z}}^{k} , \, {\varvec{v}}^{k} {)} = 0 \hfill \\ \Rightarrow \nabla_{{\varvec{\mu}}} \left\{ {h({\varvec{\mu}}) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\mu}} - \, {\mathbf{z}}^{k} + {\varvec{v}}^{k} } \right\|_{2}^{2} } \right\} = 0 \hfill \\ \Rightarrow \nabla_{{\varvec{\mu}}} \left\{ {({1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-\nulldelimiterspace} 2})\left\| {{\varvec{y}} \odot \left( {\user2{B\mu } - b_{2} {\mathbf{1}}_{n} } \right) - {\mathbf{1}}_{n} } \right\|_{2}^{2} + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\mu}} - \, {\mathbf{z}}^{k} + {\varvec{v}}^{k} } \right\|_{2}^{2} } \right\}{ = 0} \hfill \\ \Rightarrow {\varvec{B}}_{y}^{T} {\varvec{B}}_{y} {\varvec{\mu}} - {\varvec{B}}_{y}^{T} \left( {b_{2} {\varvec{y}} + {\mathbf{1}}_{n} } \right) + \rho {\varvec{\mu}} - \rho \left( {{\mathbf{z}}^{k} - {\varvec{v}}^{k} } \right) = 0 \hfill \\ \Rightarrow {\varvec{\mu}}^{k + 1} = \left( {{\varvec{B}}_{y}^{T} {\varvec{B}}_{y} + \rho {\varvec{I}}_{d} } \right)^{ - 1} \left\{ {{\varvec{B}}_{y}^{T} \left( {b_{2} {\varvec{y}} + {\mathbf{1}}_{n} } \right) + \rho \left( {{\mathbf{z}}^{k} - {\varvec{v}}^{k} } \right)} \right\}{. } \hfill \\ \end{gathered}$$

The $z{\text{ - minimization}}$ of the $\mu {\text{ - step}}$ EM²NOLC algorithm has the minimization problem with the form

$$\begin{gathered} {\mathbf{z}}^{k + 1} = \mathop {\arg \min }\limits_{{{\mathbf{z}} \in {\mathbb{R}}^{d} }} {\varvec{L}}_{\rho } {(}{\varvec{\mu}}^{k + 1} , \, {\mathbf{z}}, \, {\varvec{v}}^{k} {)} \hfill \\ { } = \mathop {\arg \min }\limits_{{{\mathbf{z}} \in {\mathbb{R}}^{d} }} \left\{ {l({\mathbf{z}}) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\mu}}^{k + 1} - {\mathbf{z}} + {\varvec{v}}^{k} } \right\|_{2}^{2} } \right\}. \hfill \\ \end{gathered}$$

Similarly, owing to $S_{0} (C_{\mu } ) = \{ {\mathbf{z}} \in {\mathbb{R}}^{d} |\left\| {\mathbf{z}} \right\|_{0} \le C_{\mu } \}$ is a nonconvex set, we can apply the projected gradient method to the $z{\text{ - minimization}}$ to obtain the $z{\text{ - update}}$:

$$\begin{gathered} {\mathbf{z}}^{k + 1} = \mathop {\arg \min }\limits_{{{\mathbf{z}} \in {\mathbb{R}}^{d} }} \left\{ {I_{{S_{0} (C_{\mu } )}} ({\mathbf{z}}) + \left( {{\rho \mathord{\left/ {\vphantom {\rho 2}} \right. \kern-\nulldelimiterspace} 2}} \right)\left\| {{\varvec{\mu}}^{k + 1} - {\mathbf{z}} + {\varvec{v}}^{k} } \right\|_{2}^{2} } \right\} \hfill \\ \, \approx \mathop {\arg \min }\limits_{{{\mathbf{z}} \in S_{0} (C_{\mu } )}} \left\| {{\varvec{\mu}}^{k + 1} + {\varvec{v}}^{k} - {\mathbf{z}}} \right\|_{2}^{2} \hfill \\ \, = \Pi_{{S_{0} (C_{\mu } )}} ({\varvec{\mu}}^{k + 1} + {\varvec{v}}^{k} ), \hfill \\ \end{gathered}$$

with the indicator function $I_{{S_{0} (C_{\mu } )}} ({\mathbf{z}}) = 1$ for ${\mathbf{z}} \in S_{0} (C_{\mu } )$ and $I_{{S_{0} (C_{\mu } )}} ({\mathbf{z}}) = 0$ for ${\mathbf{z}} \notin S_{0} (C_{\mu } )$. For any vector ${\mathbf{z}}$(${\mathbf{z}} \in {\mathbb{R}}^{d}$), the projection operator $\Pi_{{S_{0} (C_{\mu } )}} ({\mathbf{z}})$ can be carried out by sorting the elements of the vector ${\mathbf{z}}$ in descending order of their absolute values and setting all elements to zeros except the largest $C_{\mu }$ elements.

Finally, the $v{\text{ - update}}$ step can be regarded as the change of constraint residuals in the ADMM problem (20), which guarantees the convergence of the ADMM updates (24), (25), and (23).

Appendix B

2.1 KS curves in Fig. 2

See Fig. 2.

Fig. 2

KS curves of four ADMM classifiers on four medical test sets (the upper sold lines for AP and the lower dot lines for NP)

2.2 ROC curves in Fig. 3

See Fig. 3.

Fig. 3

ROC curves of four ADMM classifiers on four medical test sets

2.3 IFs for EM2NOLC in Fig. 4

See Fig. 4.

Fig. 4

Important instances (IIs) with their percentage values of II (%) given by EM²NOLC on four training sets

2.4 IFs for EM2NOLC in Fig. 5

See Fig. 5.

Fig. 5

Important features (IFs) with their percentage values of FI (%) given by EM²NOLC on four training sets

2.5 Correlation analysis of selected features in Fig. 6

See Fig. 6.

Fig. 6

Correlation analysis of selected features across folds for the GC dataset

2.6 Correlation analysis of selected features in Fig. 7

See Fig. 7.

Fig. 7

Correlation analysis of selected features across folds for the MSJ dataset