Globally Optimal Linear Model Fitting with Unit-Norm Constraint

Abstract

Robustly fitting a linear model from outlier-contaminated data is an important and basic task in many scientific fields, and it is often tackled by consensus set maximization. There have been several studies on globally optimal methods for consensus set maximization, but most of them are currently confined to problems with small number of input observations and low outlier ratios. In this paper, we develop a globally optimal algorithm aiming at consensus set maximization to solve the robust linear model fitting problems with the unit-norm constraint, which is based on the branch-and-bound optimization framework. The unit-norm constraint is utilized to fix the unknown scale of linear model parameters, and we propose a compact representation of the unit-bounded searching domain to avoid introducing the additional non-linearity in the unit-norm constraint. The compact representation leads to a geometrically derived bound, which accelerates the calculation and enables the method to handle the problems with large number of observations. Experiments on both synthetic and real data show that the proposed algorithm outperforms existing globally optimal methods, especially in low dimensional problems with large number of input observations and high outlier ratios. The implementation of the source code is publicly available https://github.com/YiruWangYuri/Demo-for-GoCR.

Appendix: Proof of Lemma 1

Lemma 2

$\varvec{x}_p\in \mathbb {S}^{d+}$, $\varvec{x}_q \in \mathbb {S}^{d+}$ are two vectors in the d-dimensional unit hyper-hemisphere, and $\varvec{\varphi }_p \in \mathbb {R}^d$, $\varvec{\varphi }_q \in \mathbb {R}^d$ are the two corresponding points in the d-dimensional solid ball.

$$\begin{aligned} \angle (\varvec{x}_p,\varvec{x}_q)\le \Vert \varvec{\varphi }_p-\varvec{\varphi }_q\Vert \end{aligned}$$

(28)

Proof

$\varvec{x}_p, \varvec{x}_q, \varvec{\varphi }_p, \varvec{\varphi }_q$ can be expressed as:

$$\begin{aligned} \varvec{\varphi }_p&= \varvec{\omega }_p \cdot \theta _p=[\omega _{p1}\theta _p, \omega _{p2}\theta _p,\cdots , \omega _{pd}\theta _p]^T \end{aligned}$$

(29)

$$\begin{aligned} \varvec{\varphi }_q&= \varvec{\omega }_q \cdot \theta _q=[\omega _{q1}\theta _q, \omega _{q2}\theta _q,\cdots , \omega _{qd}\theta _q]^T \end{aligned}$$

(30)

$$\begin{aligned} \varvec{x}_p&= [\cos (\theta _p), \omega _{p1} \sin (\theta _p), \omega _{p2} \sin (\theta _p),\cdots , \omega _{pd} \sin (\theta _p)]^T \end{aligned}$$

(31)

$$\begin{aligned} \varvec{x}_q&= [\cos (\theta _q), \omega _{q1} \sin (\theta _q), \omega _{q2} \sin (\theta _q),\cdots , \omega _{qd} \sin (\theta _q)]^T \end{aligned}$$

(32)

Since $\Vert \varvec{x}_p \Vert = \Vert \varvec{x}_q \Vert = \Vert \varvec{\omega }_p \Vert = \Vert \varvec{\omega }_q \Vert = 1$, we can define $\gamma =\angle (\varvec{\omega }_p,\varvec{\omega }_q)=\arccos (\frac{\varvec{\omega _p^T\omega _q}}{\Vert \varvec{\omega }_p \Vert \Vert \varvec{\omega }_q \Vert } )=\arccos (\varvec{\omega _p^T\omega _q})$.

On the left, we have

$$\begin{aligned} \rho&= \angle (\varvec{x}_p,\varvec{x}_q) \end{aligned}$$

(33)

$$\begin{aligned}&= \arccos \left( \frac{\varvec{x}_p^T \varvec{x}_q}{\Vert \varvec{x}_p \Vert \Vert \varvec{x}_q \Vert } \right) = \arccos (\varvec{x}_p^T \varvec{x}_q)\end{aligned}$$

(34)

$$\begin{aligned}&= \arccos \big (\cos (\theta _p)\cos (\theta _q)+\omega _{p1}\omega _{q1}\sin (\theta _p)\sin (\theta _q)+ \nonumber \\&\qquad \cdots +\omega _{pd}\omega _{qd}\sin (\theta _p)\sin (\theta _q)\big )\end{aligned}$$

(35)

$$\begin{aligned}&=\arccos \big (\cos (\theta _p)\cos (\theta _q)+(\varvec{\omega _p^T\omega _q}) \sin (\theta _p)\sin (\theta _q)\big )\end{aligned}$$

(36)

$$\begin{aligned}&=\arccos \big (\cos (\theta _p)\cos (\theta _q)+\cos (\gamma ) \sin (\theta _p)\sin (\theta _q)\big ) \end{aligned}$$

(37)

Let $\lambda \in [0,1]$, $\cos (\gamma )= 2\lambda -1$, then we have

$$\begin{aligned} \rho ^2&=\arccos ^2\big (\cos (\theta _p)\cos (\theta _q)+(2\lambda -1) \sin (\theta _p)\sin (\theta _q)\big )\end{aligned}$$

(38)

$$\begin{aligned}&=\arccos ^2\big (\lambda \cos (\theta _p-\theta _q)+(1-\lambda )\cos (\theta _p+\theta _q)\big ) \end{aligned}$$

(39)

On the right,

$$\begin{aligned} \xi&= \Vert \varvec{\varphi _p}-\varvec{\varphi _q}\Vert = \Vert \varvec{\omega _p}\theta _p-\varvec{\omega _q}\theta _q\Vert \end{aligned}$$

(40)

$$\begin{aligned}&=\sqrt{(\omega _{p1}\theta _p-\omega _{q1}\theta _q)^2 + \cdots + (\omega _{pd}\theta _p-\omega _{qd}\theta _q)^2}\end{aligned}$$

(41)

$$\begin{aligned}&=\sqrt{\theta _p^2+\theta _q^2-2\theta _p\theta _q \cos (\gamma )} \end{aligned}$$

(42)

Since $\cos (\gamma )= 2\lambda -1$,

$$\begin{aligned} \xi ^2&= \theta _p^2+\theta _q^2-2\theta _p\theta _q \cos (\gamma )\end{aligned}$$

(43)

$$\begin{aligned}&=\theta _p^2+\theta _q^2-2\theta _p\theta _q (2\lambda -1)\end{aligned}$$

(44)

$$\begin{aligned}&=\lambda (\theta _p-\theta _q)^2+(1-\lambda )(\theta _p+\theta _q)^2 \end{aligned}$$

(45)

Fig. 7

$f(\alpha )=\arccos ^2(\alpha )$

To compare $\rho$ and $\xi$, we introduce a convex function $f(\alpha )=\arccos ^2(\alpha )$, when $\alpha \in [-1,1]$, see Fig. 7. According to Jensen’s inequality, we then have

$$\begin{aligned} f\big (\lambda \alpha _1+ (1-\lambda )\alpha _2\big )\le \lambda f(\alpha _1)+(1-\lambda )f(\alpha _2) \end{aligned}$$

(46)

Let $\alpha _1=\cos (\theta _p-\theta _q)$ and $\alpha _2=\cos (\theta _p+\theta _q)$, then we have

$$\begin{aligned} \rho ^2&=\arccos ^2(\lambda \cos (\theta _p-\theta _q)+(1-\lambda )\cos (\theta _p+\theta _q))\end{aligned}$$

(47)

$$\begin{aligned}&\le \lambda \arccos ^2(\cos (\theta _p-\theta _q))\nonumber \\&\quad +(1-\lambda )\arccos ^2(\cos (\theta _p+\theta _q)) \end{aligned}$$

(48)

$$\begin{aligned}&=\lambda (\theta _p-\theta _q)^2+(1-\lambda )(\theta _p+\theta _q)^2\end{aligned}$$

(49)

$$\begin{aligned}&= \xi ^2 \end{aligned}$$

(50)

Because of the non-negativeness of $\rho$ and $\xi$, then $\angle (\varvec{x}_p,\varvec{x}_q)\le \Vert \varvec{\varphi }_p-\varvec{\varphi }_q\Vert$ is finally proved.