Multiclass multiple kernel learning using hypersphere for pattern recognition

Abstract

Confronting with plenty of information from multiple targets and multiple sources, it is core and important to learn and decide for the bionic brain or robot within limited capacity in the bionic-technology field. Several multiclass multiple kernel learning algorithms are proposed instead of a single one, which can not only combine multiple kernels corresponding to different notions of similarity or information from multiple feature subsets, but also avoid kernel parameters selecting and fuse distinctions of multiple kernels. The core of these algorithms is the small sphere and large margin approach with hypersphere boundary, which takes the advantages of support vector machine (SVM) and support vector data description (SVDD), making the volume of sphere as small as well and the margin as large as possible, in other words, minimizing the within-class divergence like SVDD and maximizing the between-class margin like SVM. Meanwhile, The one-class essence of SSLM can relieve problem of the data imbalance. Besides, the one-against-all strategy is adopted for multiclass recognition. Hence, there will be a remarkable improvement of recognition accuracy. Numerical experiments based on three publicly UCI datasets demonstrate that using multiple kernels instead of a single one is useful and promising. These MMKL algorithms are ideal for classification and recognition of multiple targets and sources in artificial intelligence field.

Appendix

1.1 Proof of (17)

Proof

The solution of coefficients can be rewritten as ${{\eta }^{*}}=\underset {{{\left \| \eta \right \|}_{2}}= 1}{\mathop {\arg \max }}\,\frac {{{\left ({{\eta }^{T}}a \right )}^{2}}}{{{\eta }^{T}}M\eta }$.

Let u = M ^1/2 η,u ^∗ = M ^1/2 η ^∗,then

$$\begin{array}{@{}rcl@{}} {{\eta }^{*}}&=&\underset{{{\left\| \eta \right\|}_{2}}= 1}{{\arg \max }}\,\frac{{{\eta }^{T}}a{{a}^{T}}\eta }{{{\eta }^{T}}M\eta }\Rightarrow {{u}^{*}}\\ &=&\underset{{{\left\| {{M}^{-{1}/{2}\;}}u \right\|}_{2}}= 1}{{\arg \max }}\,\frac{{{u}^{T}}\left[ {{M}^{{-1}/{2}\;}}a{{a}^{T}}{{M}^{{-1}/{2}\;}} \right]u}{{{u}^{T}}u} \end{array} $$

(35)

Rewrite (35) as:

$$\begin{array}{@{}rcl@{}} {{u}^{*}}&=&\underset{{{\left\| {{M}^{-{1}/{2}\;}}u \right\|}_{2}}= 1}{{\arg \max }}\,\frac{{{\left[ {{u}^{T}}{{M}^{{-1}/{2}\;}}a \right]}^{2}}}{{{\left\| u \right\|}^{2}}}\\ &=&\underset{{{\left\| {{M}^{-{1}/{2}\;}}u \right\|}_{2}}= 1}{{\arg \max }}\,{{\left[ {{\left( \frac{u}{\left\| u \right\|} \right)}^{T}}{{M}^{{-1}/{2}\;}}a \right]}^{2}} \end{array} $$

(36)

Hence, u ^∗∈ V e c(M ^− 1/2 a),with ∥M ^− 1/2 u ^∗∥₂ = 1, whichleads to ${{\eta }^{*}}=\frac {{{M}^{{-1}/{2}\;}}a}{\left \| {{M}^{{-1}/{2}\;}}a \right \|}$.□