Multi-metric learning by a pair of twin-metric learning framework

Abstract

Multi-metric learning is important for improving classification performance since learning a single metric is usually insufficient for complex data. The existing multi-metric learning methods are based on the triplet constraints, and thus are with high computing complexity. In this work, we propose an efficient multi-metric learning framework by a pair of two-metric learning schemes (called TMML) to jointly train two local metrics and a global metric, where the distances between samples are automatically adjusted to maximize classification margin. Instead of the triplet constraints, the proposed TMML is based on the pair constraints to reduce the computational burden. Moreover, a global regularization is introduced to improve generalization and control overfitting. The proposed TMML improves the limitation of a single metric, where a pair of local metrics are interrelated to conduct adaptation for the local characteristics, while global metrics are to depict the common properties from all the data. Furthermore, we develop an alternating direction iterative algorithm to optimize the proposed TMML. The convergence of the algorithm is analyzed theoretically. Numerical experiments are carried out on different scale datasets. Under different evaluation criteria, experiments show that the proposed TMML is superior to the single metric learning methods, and achieves better performance than other state-of-the-art multi-metric learning methods in most cases.

Appendix:

Conditions to guarantee the convergence of block-coordinate descent method [27].

Assume that the objective function to be optimized has the following form:

$$ \begin{array}{@{}rcl@{}} f(x_{1},\cdots,x_{n})=f_{0}(x_{1},\cdots,x_{n})+\sum\limits_{k=1}^{n}f_{k}(x_{k}) \end{array} $$

(42)

Suppose that f,f₀,f₁,⋯ ,f_N satisfy Assumptions B1-B3 and that f₀ satisfies either the assumption C1 or C2. Also, assume that the sequence $\{x^{r}=({x_{1}^{r}},\cdots ,{x_{N}^{r}})\}_{r=0,1,\cdots }$ generated by the BCD method using the essentially cyclic rule is defined. Then, either $\{f(x^{r})\}\downarrow -\infty$, or else every cluster point z = (z₁,⋯ ,z_N) is a coordinatewise minimum point of f.

• (B1) f₀ is continuous on dom f₀.

• (B2) For each k ∈{1,⋯ ,N} and (x_j)_j≠k, the function $x_{k}\rightarrow f(x_{1},\cdots ,x_{N})$ is quasiconvex and hemivariate.

• (B3) f₀,f₁,⋯ ,f_N are lower semicontinuous.

• (C1) dom f₀ is open and f₀ tends to $\infty$ at every boundary point of dom f₀.

• (C2) dom f₀ = Y₁×,⋯ ,×Y_N, for some $Y_{k} \subseteq R^{n_{k}},k=1,\cdots ,N$

Theorem

[27]. Suppose that f,f₀,⋯ ,f_N satisfy assumptions B1-B3 and that f₀ satisfies either assumption C1 or C2. Using the essentially cyclic, the block-coordinate descent method converges to an optimal point of f.