Exploiting Privileged Information from Web Data for Action and Event Recognition

Abstract

In the conventional approaches for action and event recognition, sufficient labelled training videos are generally required to learn robust classifiers with good generalization capability on new testing videos. However, collecting labelled training videos is often time consuming and expensive. In this work, we propose new learning frameworks to train robust classifiers for action and event recognition by using freely available web videos as training data. We aim to address three challenging issues: (1) the training web videos are generally associated with rich textual descriptions, which are not available in test videos; (2) the labels of training web videos are noisy and may be inaccurate; (3) the data distributions between training and test videos are often considerably different. To address the first two issues, we propose a new framework called multi-instance learning with privileged information (MIL-PI) together with three new MIL methods, in which we not only take advantage of the additional textual descriptions of training web videos as privileged information, but also explicitly cope with noise in the loose labels of training web videos. When the training and test videos come from different data distributions, we further extend our MIL-PI as a new framework called domain adaptive MIL-PI. We also propose another three new domain adaptation methods, which can additionally reduce the data distribution mismatch between training and test videos. Comprehensive experiments for action and event recognition demonstrate the effectiveness of our proposed approaches.

Appendices

Appendix 1: Detailed Derivations for (16)

We provide the complete derivations for (16). For ease of presentation, we define

$$\begin{aligned} F(\hat{\varvec{\alpha }}, \varvec{\beta }) = \frac{1}{2\gamma }(\hat{\varvec{\alpha }}+ \varvec{\beta }- C_1\mathbf {1})'\tilde{\mathbf {K}}(\hat{\varvec{\alpha }}+ \varvec{\beta }- C_1\mathbf {1}). \end{aligned}$$

Then, the problem in (15) can be rewritten as,

$$\begin{aligned}&\min _{\mathbf {d}} \max _{(\varvec{\alpha },\varvec{\beta }) \in \mathcal {S}} \quad \mathbf {1}'\varvec{\alpha }-\frac{1}{2} \varvec{\alpha }'\left( \sum _{t=1}^Td_t\mathbf {Q}\circ \mathbf {y}_t\mathbf {y}_t'\right) \varvec{\alpha }- F(\hat{\varvec{\alpha }}, \varvec{\beta })\nonumber \\&\quad \text{ s.t. } \quad \sum _{t=1}^Td_t = 1, \quad d_t \ge 0, \quad \forall t = 1,\ldots , T \end{aligned}$$

(44)

Let us introduce a dual variable $\tau$ for the constraint $\sum _{t=1}^Td_t = 1$ and another dual variable $\nu _t$ for each constraint $d_t \ge 0$ in problem (44), we arrive at its Lagrangian as follows,

$$\begin{aligned} \mathcal {L}= & {} \mathbf {1}'\varvec{\alpha }-\frac{1}{2} \varvec{\alpha }'\left( \sum _{t=1}^Td_t\mathbf {Q}\circ \mathbf {y}_t\mathbf {y}_t'\right) \varvec{\alpha }-F(\hat{\varvec{\alpha }}, \varvec{\beta })\nonumber \\&+\, \tau \left( \sum _{t=1}^Td_t - 1\right) - \sum _{t=1}^T \nu _td_t. \end{aligned}$$

(45)

The derivative of the Lagrangian w.r.t. $d_t$ can be written as,

$$\begin{aligned} \frac{\partial \mathcal {L}}{\partial d_t} = -\frac{1}{2} \varvec{\alpha }'\left( \mathbf {Q}\circ \mathbf {y}_t\mathbf {y}_t'\right) \varvec{\alpha }+ \tau - \nu _t, \quad \forall t = 1, \ldots , T. \end{aligned}$$

Let us set $\frac{\partial \mathcal {L}}{\partial d_t} = 0$ and consider $\nu _t \ge 0$, we have

$$\begin{aligned} \frac{1}{2} \varvec{\alpha }'\left( \mathbf {Q}\circ \mathbf {y}_t\mathbf {y}_t'\right) \varvec{\alpha }\le \tau , \quad \forall t = 1, \ldots , T. \end{aligned}$$

By substituting $\frac{\partial \mathcal {L}}{\partial d_t} = 0$ into the Lagrangian, we obtain the duality of (44) as follows,

$$\begin{aligned}&\max _{\tau }\max _{(\varvec{\alpha },\varvec{\beta }) \in \mathcal {S}} \quad 1'\varvec{\alpha }- F(\hat{\varvec{\alpha }}, \varvec{\beta }) - \tau \nonumber \\&\quad \text{ s.t. } \quad \frac{1}{2} \varvec{\alpha }'\left( \mathbf {Q}\circ \mathbf {y}_t\mathbf {y}_t'\right) \varvec{\alpha }\le \tau , \quad \forall t = 1, \ldots , T, \end{aligned}$$

(46)

which is the same as (16). We complete the derivations here.

Appendix 2: Solution to the MKL Problem at Step 4 of Algorithm 3

At Step 4 of Algorithm 3, we solve an MKL problem in (15) by setting $\mathcal {Y}= \mathcal {C}$. As $\mathcal {C}$ contains only a small number of label vectors, so the number of base kernels is not large. Now we give the algorithm for solving the MKL problem in (15) with a few kernels.

Let us denote $\tilde{T} = |\mathcal {C}|$ as the number of label vectors in $\mathcal {C}$. We also define $\mathbf {d}= [d_1, \ldots , d_{\tilde{T}}]'$ as the vector of kernel coefficients, and $\mathcal {D}= \{\mathbf {d}'\mathbf {1} = 1, \mathbf {d}\ge 0\}$. Note we use the same symbols $\mathbf {d}$ and $\mathcal {D}$ as in (15) for simplicity, but the dimensionality of $\mathbf {d}$ (i.e., $\tilde{T} = |\mathcal {C}|$) is much smaller than that in (15) (i.e., $T = |\mathcal {Y}|$). Now, we write the primal form of (15) as follows,

$$\begin{aligned}&\min _{\mathop {\tilde{\mathbf {w}}, \tilde{b}, \mathbf {w}_t, \varvec{\eta }}\limits ^{\mathbf {d}\in \mathcal {D}}}\quad \frac{1}{2}\sum _{t=1}^{\tilde{T}}\frac{\Vert \mathbf {w}_t\Vert ^2}{d_t}+\frac{\gamma }{2}\Vert \tilde{\mathbf {w}}\Vert ^2\nonumber \\&\qquad + \,C_1\sum _{i=1}^{n^+}\xi (\tilde{\phi }(\tilde{\mathbf {x}}_i)) + \sum _{i=n^+ +1}^{n}\eta _i, \end{aligned}$$

(47)

$$\begin{aligned}&\quad \text{ s.t. } \quad \sum _{t=1}^{\tilde{T}}\mathbf {w}_t'\psi _t(\mathbf {x}_i)\ge 1 - \xi (\tilde{\phi }(\tilde{\mathbf {x}}_i)),\nonumber \\&\qquad \qquad \xi (\tilde{\phi }(\tilde{\mathbf {x}}_i)) \ge 0, \qquad i=1,\ldots ,n^+, \nonumber \\&\qquad \qquad \sum _{t=1}^{\tilde{T}}\mathbf {w}_t'\psi _t(\mathbf {x}_i) \ge 1 - \eta _i,\nonumber \\&\qquad \qquad \eta _i \ge 0, \qquad i=n^+ +1, \ldots , n, \end{aligned}$$

(48)

where $\psi _t(\mathbf {x}_i)$ is the nonlinear feature of $\mathbf {x}_i$ induced by the kernel $\mathbf {Q}\circ \mathbf {y}_t\mathbf {y}_t'$, and $\mathbf {w}_t = d_t\sum _{i=1}^{n}\alpha _iy^t_i\phi (\mathbf {x}_i)$ with $y^t_i$ being the i-the entry in $\mathbf {y}_t$.

The above problem is a convex problem w.r.t. $\tilde{\mathbf {w}}, \tilde{b}$, $\mathbf {w}_t, \varvec{\eta }$, and $\mathbf {d}$, so we can achieve the global optimum by alternatively optimizing two set of variables $\{\tilde{\mathbf {w}}, \tilde{b}, \mathbf {w}_t, \varvec{\eta }\}$, and $\mathbf {d}$.

Fix $\mathbf {d}$: When $\mathbf {d}$ is fixed, we solve for $\{\tilde{\mathbf {w}}, \tilde{b}, \mathbf {w}_t, \varvec{\eta }\}$ by optimizing the dual problem in (15), i.e.,

$$\begin{aligned}&\max _{(\varvec{\alpha },\varvec{\beta }) \in \mathcal {S}} \qquad \mathbf {1}'\varvec{\alpha }-\frac{1}{2} \varvec{\alpha }'\left( \sum _{t=1}^{\tilde{T}}d_t\mathbf {Q}\circ \mathbf {y}_t\mathbf {y}_t'\right) \varvec{\alpha }\nonumber \\&\quad -\,\frac{1}{2\gamma }(\hat{\varvec{\alpha }}+ \varvec{\beta }- C_1\mathbf {1})'\tilde{\mathbf {K}}(\hat{\varvec{\alpha }}+ \varvec{\beta }- C_1\mathbf {1}), \end{aligned}$$

(49)

which is a quadratic programming problem w.r.t. $(\varvec{\alpha }, \varvec{\beta })$, and can be solved by using any QP solver.

Fix $\{\tilde{\mathbf {w}}, \tilde{b}, \mathbf {w}_t, \varvec{\eta }\}$: The optimization problem w.r.t. $\mathbf {d}$ can be written as,

$$\begin{aligned}&\min _{\mathbf {d}} \quad \frac{1}{2}\sum _{t=1}^{\tilde{T}}\frac{\Vert \mathbf {w}_t\Vert ^2}{d_t}\nonumber \\&\quad \text{ s.t. } \quad \mathbf {d}'\mathbf {1} = 1, \quad \mathbf {d}\ge 0, \end{aligned}$$

(50)

which is the same as solving the kernel coefficients in $\ell _p$-norm MKL (Kloft et al. 2011) when $p = 1$, and has a closed-form solution as below,

$$\begin{aligned} d_t = \frac{\Vert \mathbf {w}_t\Vert }{\sum _{t=1}^{\tilde{T}}\Vert \mathbf {w}_t\Vert }, \end{aligned}$$

(51)

where $\Vert \mathbf {w}_t\Vert$ can be calculated from $\Vert \mathbf {w}_t\Vert ^2 = d_t^2\varvec{\alpha }'(\mathbf {Q}\circ \mathbf {y}_t\mathbf {y}_t')\varvec{\alpha }$. We repeat above two steps until the objective value of (49) converges.

Appendix 3: Proof of Proposition 1

Proof

By introducing the dual variables $\hat{\varvec{\alpha }} = [\alpha _1, \ldots , \alpha _{L^+}]' \in {\mathbb R}^{L^+}$ for the constraints in (23), $\bar{\varvec{\alpha }} = [\alpha _{L^++1}, \ldots , \alpha _m]' \in {\mathbb R}^{m-L^+}$ for the constraints (24), $\hat{\varvec{\beta }} = [\beta _{1}, \ldots , \beta _{L^+}]' \in {\mathbb R}^{L^+}$ for the constraints in (25), $\bar{\varvec{\beta }} = [\beta _{L^++1}, \ldots , \beta _{m}]' \in {\mathbb R}^{m-L^+}$ for the constraints in (26), and ${\varvec{\nu }}= [\nu _1, \ldots , \nu _m]'$ for the constraints in (27), we arrive at its Lagrangian as follows:

$$\begin{aligned} \mathcal {L}= & {} \frac{1}{2}\left( \Vert \mathbf {w}\Vert ^2+\gamma \Vert \tilde{\mathbf {w}}\Vert ^2\right) + C_1\sum _{i=1}^{L^+}{(\tilde{\mathbf {w}}'\tilde{\mathbf {z}}^s_i+\tilde{b})} \nonumber \\&+ \sum _{i=L^+ +1}^{m}{\eta _i} + \frac{\lambda }{2}\Vert \hat{\mathbf {w}} - \rho \mathbf {v}\Vert ^2 + C_2\sum _{i=1}^{m}(\hat{\mathbf {w}}'\mathbf {z}^s_i + \hat{b})\nonumber \\&- \sum _{i=1}^{L^+} \hat{\alpha _i}(\mathbf {w}'\mathbf {z}^s_i + b - p_i + \tilde{\mathbf {w}}'\tilde{\mathbf {z}}^s_i+\tilde{b} + \hat{\mathbf {w}}'\mathbf {z}^s_i + \hat{b}) \nonumber \\&- \sum _{i=L^+ +1}^{m} \bar{\alpha _i}(-\mathbf {w}'\mathbf {z}^s_i - b -1 + \eta _i + \hat{\mathbf {w}}'\mathbf {z}^s_i + \hat{b}) \nonumber \\&- \sum _{i=1}^{L^+} \hat{\beta _i}(\tilde{\mathbf {w}}'\tilde{\mathbf {z}}^s_i+\tilde{b}) - \sum _{i=L^+ +1}^{m} \bar{\beta _i}\eta _i - \sum _{i=1}^{m} \nu _i (\hat{\mathbf {w}}'\mathbf {z}^s_i + \hat{b}),\nonumber \\ \end{aligned}$$

(52)

Let us define $\varvec{\alpha }= [\hat{\varvec{\alpha }}', \bar{\varvec{\alpha }}']'$, $\varvec{\beta }= [\hat{\varvec{\beta }}', \bar{\varvec{\beta }}']'$, $\mathbf {Z}= [\mathbf {z}^s_1, \ldots , \mathbf {z}^s_m]$, $\tilde{\mathbf {Z}}= [\tilde{\mathbf {z}}^s_1, \ldots , \tilde{\mathbf {z}}^s_{L^+}]$, and $\mathbf {y}= [\mathbf {1}_{L^+}', -\mathbf {1}_{m-L^+}']'$, then the derivatives of the Lagrangian w.r.t. $\mathbf {w},b,\tilde{\mathbf {w}},\tilde{b},\hat{\mathbf {w}},\hat{b}, \varvec{\eta }$ can be obtained as follows:

$$\begin{aligned} \frac{\partial \mathcal {L}}{\partial \mathbf {w}}= & {} \mathbf {w}-\mathbf {Z}(\varvec{\alpha }\circ \mathbf {y}),\nonumber \\ \frac{\partial \mathcal {L}}{\partial b}= & {} -\varvec{\alpha }'\mathbf {y},\nonumber \\ \frac{\partial \mathcal {L}}{\partial \tilde{\mathbf {w}}}= & {} \gamma \tilde{\mathbf {w}}-\tilde{\mathbf {Z}}(\hat{\varvec{\alpha }}+\hat{\varvec{\beta }}-C_1\mathbf {1}_{L^+}),\nonumber \\ \frac{\partial \mathcal {L}}{\partial \tilde{b}}= & {} -\mathbf {1}_{L^+}'(\hat{\varvec{\alpha }}+\hat{\varvec{\beta }}-C_1 \mathbf {1}_{L^+}),\nonumber \\ \frac{\partial \mathcal {L}}{\partial \hat{\mathbf {w}}}= & {} \lambda \hat{\mathbf {w}}-\lambda \rho \mathbf {v}-\mathbf {Z}(\varvec{\alpha }+{\varvec{\nu }}-C_2\mathbf {1}_m),\nonumber \\ \frac{\partial \mathcal {L}}{\partial \hat{b}}= & {} -\mathbf {1}_m'(\varvec{\alpha }+{\varvec{\nu }}-C_2\mathbf {1}),\nonumber \\ \frac{\partial \mathcal {L}}{\partial \varvec{\eta }}= & {} \mathbf {1}_{m-L^+}-\bar{\varvec{\alpha }} - \bar{\varvec{\beta }}\nonumber . \end{aligned}$$

By setting those derivatives to zeros, we have the following equations:

$$\begin{aligned} \mathbf {w}= & {} \mathbf {Z}(\varvec{\alpha }\circ \mathbf {y}),\end{aligned}$$

(53)

$$\begin{aligned} \tilde{\mathbf {w}}= & {} \frac{1}{\gamma }\tilde{\mathbf {Z}}(\hat{\varvec{\alpha }}+\hat{\varvec{\beta }}-C_1\mathbf {1}_{L^+}),\end{aligned}$$

(54)

$$\begin{aligned} \hat{\mathbf {w}}= & {} \rho \mathbf {v}+ \frac{1}{\lambda }\mathbf {Z}(\varvec{\alpha }+{\varvec{\nu }}-C_2\mathbf {1}_m), \end{aligned}$$

(55)

as well as the following constraints, $\varvec{\alpha }'\mathbf {y}= 0$, $\mathbf {1}_{L^+}'(\hat{\varvec{\alpha }}+\hat{\varvec{\beta }}-C_1 \mathbf {1}_{L^+}) = 0$, $\mathbf {1}_m'(\varvec{\alpha }+{\varvec{\nu }}-C_2\mathbf {1}_m) = 0$, $\bar{\varvec{\alpha }} \le \mathbf {1}_{m-L^+}$. Substituting the equations (53), (54) and (55) into (52) and considering $\varvec{\alpha }, \varvec{\beta }, {\varvec{\nu }}\ge \mathbf {0}$, we obtain the following dual form,

$$\begin{aligned}&\min _{\varvec{\alpha }, \varvec{\beta }, {\varvec{\nu }}} \quad -\mathbf {p}'\varvec{\alpha }+ \frac{1}{2} \varvec{\alpha }'(\mathbf {K}\circ \mathbf {y}\mathbf {y}')\varvec{\alpha }\nonumber \\&\quad + \frac{1}{2\gamma }(\hat{\varvec{\alpha }}+ \hat{\varvec{\beta }} - C_1\mathbf {1})'\tilde{\mathbf {K}}(\hat{\varvec{\alpha }}+ \hat{\varvec{\beta }} - C_1\mathbf {1})\nonumber \\&\quad +\frac{1}{2\lambda }(\varvec{\alpha }+ {\varvec{\nu }}-C_2\mathbf {1}_m)'\mathbf {K}(\varvec{\alpha }+{\varvec{\nu }}- C_2\mathbf {1}_m) \nonumber \\&\quad + \rho \mathbf {v}'\mathbf {Z}(\varvec{\alpha }+{\varvec{\nu }}-C_2\mathbf {1}_m)\end{aligned}$$

(56)

$$\begin{aligned}&\quad \text{ s.t. }\quad \varvec{\alpha }'\mathbf {y}= 0, \quad \mathbf {1}_{L^+}'(\hat{\varvec{\alpha }}+\hat{\varvec{\beta }}-C_1 \mathbf {1}_{L^+}) = 0, \nonumber \\&\qquad \qquad \bar{\varvec{\alpha }} \le \mathbf {1}_{m-L^+}, \nonumber \\&\qquad \qquad \mathbf {1}_m'(\varvec{\alpha }+{\varvec{\nu }}-C_2\mathbf {1}_m) = 0, \quad \varvec{\alpha }, \varvec{\beta }, {\varvec{\nu }}\ge \mathbf {0}, \end{aligned}$$

(57)

Let us define $\varvec{\theta }= \frac{1}{C_2}(\varvec{\alpha }+ {\varvec{\nu }})$, then the constraint $\mathbf {1}_m'(\varvec{\alpha }+{\varvec{\nu }}-C_2\mathbf {1}_m) = 0$ becomes $\mathbf {1}_m'\varvec{\theta }= m$, and the constraint ${\varvec{\nu }}\ge \mathbf {0}$ becomes $\varvec{\alpha }\le C_2\varvec{\theta }$. Let us define the feasible set for $(\varvec{\alpha }, \varvec{\beta }, {\varvec{\nu }})$ as $\mathcal {A}= \{ \varvec{\alpha }'\mathbf {y}= 0, \mathbf {1}_{L^+}'(\hat{\varvec{\alpha }}+\hat{\varvec{\beta }}-C_1 \mathbf {1}_{L^+}) = 0, \bar{\varvec{\alpha }} \le \mathbf {1}_{m-L^+}, \mathbf {1}_m'\varvec{\theta }= m, \varvec{\alpha }\le C_2\varvec{\theta }, \varvec{\alpha }, \varvec{\beta }, \varvec{\theta }\ge \mathbf {0}\}$. Substituting $\varvec{\theta }= \frac{1}{C_2}(\varvec{\alpha }+ {\varvec{\nu }})$ into (56), we arrive at,

$$\begin{aligned}&\min _{(\varvec{\alpha }, \varvec{\beta }, \varvec{\theta })\in \mathcal {A}} \quad -\mathbf {p}'\varvec{\alpha }+ \frac{1}{2} \varvec{\alpha }'(\mathbf {K}\circ \mathbf {y}\mathbf {y}')\varvec{\alpha }\nonumber \\&\quad + \frac{1}{2\gamma }(\hat{\varvec{\alpha }}+ \hat{\varvec{\beta }} - C_1\mathbf {1})'\tilde{\mathbf {K}}(\hat{\varvec{\alpha }}+ \hat{\varvec{\beta }} - C_1\mathbf {1})\nonumber \\&\quad +\frac{(C_2)^2}{2\lambda }(\varvec{\theta }- \mathbf {1}_m)'\mathbf {K}(\varvec{\theta }-\mathbf {1}_m) + \rho C_2\mathbf {v}'\mathbf {Z}(\varvec{\theta }- \mathbf {1}_m) \end{aligned}$$

(58)

Recall in the main text we have defined $H(\varvec{\alpha },\varvec{\beta })=-\mathbf {p}'\varvec{\alpha }+ \frac{1}{2} \varvec{\alpha }'(\mathbf {K}\circ \mathbf {y}\mathbf {y}')\varvec{\alpha }+ \frac{1}{2\gamma }(\hat{\varvec{\alpha }}+ \hat{\varvec{\beta }} - C_1\mathbf {1})'\tilde{\mathbf {K}}(\hat{\varvec{\alpha }}+ \hat{\varvec{\beta }} - C_1\mathbf {1})$, then we simplify the objective function in (58) as follows,

$$\begin{aligned}&\min _{(\varvec{\alpha }, \varvec{\beta }, \varvec{\theta })\in \mathcal {A}} \quad H(\varvec{\alpha }, \varvec{\beta }) +\frac{(C_2)^2}{2\lambda }(\varvec{\theta }- \mathbf {1}_m)'\mathbf {K}(\varvec{\theta }-\mathbf {1}_m) \nonumber \\&\quad + \rho C_2\mathbf {v}'\mathbf {Z}(\varvec{\theta }- \mathbf {1}_m) \end{aligned}$$

(59)

Now, we derive the objective function as follows,

$$\begin{aligned}&\min _{(\varvec{\alpha }, \varvec{\beta }, \varvec{\theta })\in \mathcal {A}}\quad H(\varvec{\alpha }, \varvec{\beta }) +\frac{(C_2)^2}{2\lambda }(\varvec{\theta }- \mathbf {1}_m)'\mathbf {K}(\varvec{\theta }-\mathbf {1}_m)\nonumber \\&\quad + \rho C_2\mathbf {v}'\mathbf {Z}(\varvec{\theta }- \mathbf {1}_m)\end{aligned}$$

(60)

$$\begin{aligned}&\Leftrightarrow \min _{(\varvec{\alpha }, \varvec{\beta }, \varvec{\theta })\in \mathcal {A}}\quad H(\varvec{\alpha }, \varvec{\beta }) +\frac{(C_2)^2}{2\lambda }(\varvec{\theta }'\mathbf {K}\varvec{\theta }- 2\mathbf {1}_m'\mathbf {K}\varvec{\theta }) \nonumber \\&\quad + \rho C_2\mathbf {v}'\mathbf {Z}\varvec{\theta }\end{aligned}$$

(61)

$$\begin{aligned}&\Leftrightarrow \min _{(\varvec{\alpha }, \varvec{\beta }, \varvec{\theta })\in \mathcal {A}} \quad H(\varvec{\alpha }, \varvec{\beta }) +\frac{(C_2)^2}{2\lambda }\varvec{\theta }'\mathbf {K}\varvec{\theta }- \frac{(C_2)^2}{\lambda }\mathbf {1}_m'\mathbf {K}\varvec{\theta }\nonumber \\&\quad + \frac{\rho C_2}{m}\mathbf {1}_m'\mathbf {K}\varvec{\theta }- \frac{\rho C_2}{n_t}\mathbf {1}_{n_t}'\mathbf {K}_{ts}\varvec{\theta } \end{aligned}$$

(62)

where in (61) we omit the constant terms, and in (62) we use the equation that $\mathbf {v}'\mathbf {Z}= \frac{1}{m}\mathbf {1}_m'\mathbf {K}\ - \frac{1}{n_t}\mathbf {1}_{n_t}'\mathbf {K}_{ts}$ with $\mathbf {K}_{ts} \in {\mathbb R}^{n_t\times m}$ being the kernel matrix between the target domain samples and the source domain samples. Let us define $\lambda = \frac{(C_2m)^2}{\mu }$ and $\rho = \frac{C_2m}{\lambda } = \frac{\mu }{C_2m}$ and omit the constant term, then the problem in (62) becomes

$$\begin{aligned}&\min _{(\varvec{\alpha }, \varvec{\beta }, \varvec{\theta })\in \mathcal {A}} \quad H(\varvec{\alpha }, \varvec{\beta }) +\frac{\mu }{2m^2}\varvec{\theta }'\mathbf {K}\varvec{\theta }- \frac{\mu }{mn_t}\mathbf {1}_{n_t}'\mathbf {K}_{ts}\varvec{\theta }\nonumber \\&\Leftrightarrow \min _{(\varvec{\alpha }, \varvec{\beta }, \varvec{\theta })\in \mathcal {A}} \quad H(\varvec{\alpha }, \varvec{\beta }) +\frac{\mu }{2m^2}\varvec{\theta }'\mathbf {K}\varvec{\theta }- \frac{\mu }{mn_t}\mathbf {1}_{n_t}'\mathbf {K}_{ts}\varvec{\theta }\nonumber \\&\quad + \frac{\mu }{2n_t^2}\mathbf {1}_{n_t}'\mathbf {K}_t\mathbf {1}_{n_t}\end{aligned}$$

(63)

$$\begin{aligned}&\Leftrightarrow \min _{(\varvec{\alpha }, \varvec{\beta }, \varvec{\theta })\in \mathcal {A}} \quad H(\varvec{\alpha }, \varvec{\beta }) +\frac{\mu }{2}\Vert \frac{1}{m}\sum _{i=1}^m\theta _i\mathbf {z}^s_i - \frac{1}{n_t}\sum _{i=1}^{n_t}\mathbf {z}^t_i\Vert ^2,\nonumber \\ \end{aligned}$$

(64)

where in (63) we add a constant $\frac{\mu }{2n_t^2}\mathbf {1}_{n_t}'\mathbf {K}_t\mathbf {1}_{n_t}$ to the objective function with $\mathbf {K}_t \in {\mathbb R}^{n_t\times n_t}$ being the kernel matrix on the target domain samples. Note the problem in (64) is exactly the problem in (18). We complete the proof here. $\square$