Asymmetric kernel-based robust classification by ADMM

Abstract

Correntropy is a locally second-order statistical measure in kernel space. The different kernel functions induce different correntropy with different properties. In this work, we propose an asymmetric mixture kernel and the corresponding correntropy. Then, we propose a new loss function (called RCH-loss) that is induced by the correntropy with the reproducing asymmetric kernel. Some important properties of the proposed kernel and RCH-loss are demonstrated such as non-convexity, boundedness, asymmetry and asymptotic approximation. Moreover, the proposed RCH-loss satisfies Bayes optimal decision rule. With the RCH-loss function, a new robust classification framework is built to handle robust classification. Theoretically, we prove the generalization bound of the proposed model based on the Rademacher complexity. Following that, DC (difference of convex functions) programming algorithm (DCA) is developed to solve the problem iteratively, where ADMM (alternating direction method of multipliers) is used to solve the subproblem at each iteration. Moreover, we analyze the computation complexity of the algorithm and the sensitivity of parameters. Numerical experimentations are carried out on various datasets including benchmark data sets and artificial data sets with different noise levels. The experimental results display the feasibility and effectiveness of the proposed methods.

Appendix. DC programming and DCA

We outline the main algorithmic results for DC programming [18, 19]. The key to DC programs is to decompose an objective function into the difference of two convex functions, from which a sequence of approximations of the objective function yields a sequence of solutions converging to a stationary point, possibly an optimal solution. Generally speaking, a so-called DC program is to minimize a DC function:

$$\begin{aligned} f(x)=g(x)-h(x),x\in R^{n} \end{aligned}$$

(41)

with g(x) and h(x) being convex functions.

The DCA is an iterative algorithm based on local optimality conditions and duality [14-17]. The main scheme of DCA is: (for simplify, we omit here the dual part), at each iteration, one replaces in the primal DC problem ($P_{dc}$) the second component h by its affine minorization: $h(x^{k})+(x-x^{k})^{T}y^{k}$, to generate the convex program:

$$\begin{aligned} minimize: \{g(x)-h(x^{k})-(x-x^{k})^{T}y^{k},x\in R^{n},y^{k}\in \partial h(x^{k}) \} \end{aligned}$$

(42)

where $\partial h$ is the subdifferential of convex function h. In practice, a simplified form of the DCA is used. Two sequences $\{x^{k}\}$ and $\{y^{k}\}$ satisfying $y^{k}\in \partial h(x^{k})$ are constructed, and $x^{k+1}$ is a solution to the convex program (42). The simplified DCA is described as follows.

Initialization: Choose an initial point $x^{0}\in R^{n}$ and let $k=0$

Repeat

Calculate $y^{k}\in \partial h(x^k)$

Solve convex program (42) to obtain $x^{k+1}$

Let k:=k+1

Until some stopping criterion is satisfied.

DCA is a descent method without line search, and it converges linearly for general DC programs.