Deep Metric Learning with False Positive Probability

Abstract

In recent years, deep metric learning has been an end-to-end fashion in computer vision community due to the great success of deep learning. However, existing deep metric learning frameworks are faced with a dilemma about the hard level trade-off for training examples. Namely, the “harder” examples we feed to neural networks, the more likely we attain highly discriminative models, but the more easily neural networks get stuck into poor local minimal in practice. To fight against this dilemma, we propose a deep metric learning method with FAlse Positive ProbabilitY (FAPPY) to gradually incorporate different hard levels. Unlike mainstream deep metric learning schemes, the presented approach optimizes similarity probability distribution among training samples, instead of the similarity itself. Experimental results on CUB-200-2011, Stanford Online Products and VehicleID datasets show that our FAPPY method achieves or outperforms state-of-the-art metric learning methods on fine-grained image retrieval and vehicle re-identification tasks. Besides, the presented method has relatively low sensitivity of hyper-parameters and it requires minor changes on traditional classification networks.

Appendix A

Lemma 1.

Proof.

We compute the gradients of these pairs:

$$ \frac{{\partial P_{ij}^{\Delta } }}{{\partial {\text{s}}_{ij} }} = h_{i}^{t} + h_{j}^{t} , \quad \frac{{\partial P_{ij}^{\Delta } }}{{\partial {\text{s}}_{ik} }} = \left\{ {\begin{array}{*{20}c} { - h_{ij}^{right} ,} & {s_{ik} \in \left[ {b_{t} ,b_{t + 1} } \right]} \\ {0,} & {otherwise} \\ \end{array} } \right.,\quad \frac{{\partial P_{ij}^{\Delta } }}{{\partial {\text{s}}_{jl} }} = \left\{ {\begin{array}{*{20}c} { - h_{ij}^{right} , } & {s_{jl} \in \left[ {b_{t} ,b_{t + 1} } \right]} \\ {0,} & {otherwise} \\ \end{array} } \right. $$

Strictly speaking, the necessary and sufficient condition of $\frac{{\partial P_{ij}^{\Delta } }}{{\partial {\text{s}}_{ik} }} \ne 0$ is $s_{ik} , s_{ij} \in \left( {b_{t} ,b_{t + 1} } \right].$ Likewise, $\frac{{\partial P_{ij}^{\Delta } }}{{\partial {\text{s}}_{jl} }} \ne 0$ if and only if $s_{jl} , s_{ij} \in \left( {b_{t} ,b_{t + 1} } \right].$

The Equivalency

Proof.

In the L2-normalized space, the feature vectors are defined as: $\overrightarrow {{f_{x}^{L2} }} = \frac{{\overrightarrow {{f_{x} }} }}{{\overrightarrow {{\left\| {f_{x} } \right\|}} }}.$

The Euclidian distance ${\text{d}}_{xy}^{L2}$ and $\Delta^{L2}$ are as follows:

$$ {\text{d}}_{xy}^{L2} = \left\| {\overrightarrow {{f_{x}^{L2} }} - \overrightarrow {{f_{y}^{L2} }} } \right\| = \left\| {\frac{{\overrightarrow {{f_{x} }} }}{{\overrightarrow {{\left\| {f_{x} } \right\|}} }} - \frac{{\overrightarrow {{f_{y} }} }}{{\left\| {\overrightarrow {{f_{y} }} } \right\|}}} \right\| = \sqrt {1 - 2cos\left\langle {\overrightarrow {{f_{x} }} , \overrightarrow {{f_{y} }} } \right\rangle + 1} = \sqrt {2 - 2s_{xy} } $$

$$ \Delta^{L2} = d_{t}^{L2} - d_{t + 1}^{L2} = \sqrt {2 - 2b_{t} } - \sqrt {2 - 2b_{t + 1} } $$

Therefore, $\Delta^{L2}$ is positively related to $\Delta$ while ${\text{d}}_{xy}^{L2}$ is negatively related to $s_{xy}$, and the explanation in Fig. 4 is reasonable.