Plane-Based Resection for Metric Affine Cameras

Abstract

We study the problem of resecting the metric affine camera models from at least three non-colinear point correspondences. A direct application is plane pose estimation. We consider the three most popular metric affine cameras, namely the paraperspective, weak-perspective and orthographic cameras. For each model, we give an algebraic procedure which finds the optimal solution, where by optimal we mean the global minimizer of the reprojection error under the Euclidean norm. Our algebraic procedures cover both the minimal case of three points and the redundant cases of more than three points. They always return two solutions, as the problem has a two-way ambiguity on the rotation and translation for the three cameras in the general case. The scale of the paraperspective and weak-perspective cameras is, however, recovered uniquely. The orthographic case is the most involved and has not been solved analytically in the literature. We characterize its intrinsic complexity by showing that it reduces to finding the roots of an irreducible and non-solvable by radicals sextic polynomial. The previous algorithms for the paraperspective and weak-perspective cases have singularities, while, in contrast, our algebraic procedures do not.

Appendices

Translationless Formulation

We show how to eliminate the translation from the problem’s initial formulation (1). We compute the centre of gravity of the model points as:

$$\begin{aligned} \mathbf {x} {\mathop {=}\limits ^{\text {def}}}\frac{1}{m} \mathtt X \mathbf {1}. \end{aligned}$$

We then define a new set of model points by centring the initial ones, meaning that we translate their centre of gravity to the origin:

$$\begin{aligned} \mathtt X ' {\mathop {=}\limits ^{\text {def}}}\mathtt X - \mathbf {x}\mathbf {1}^\top . \end{aligned}$$

Defining the translation for the new set of model points as $\mathbf {t}' {\mathop {=}\limits ^{\text {def}}}\mathbf {t} + \mathtt P \mathbf {x}$, the problem is rewritten as:

$$\begin{aligned}&\min _{\begin{array}{c} \mathtt P \in \mathbb {M}_*\\ \mathbf {t}'\in \mathbb {R} ^2 \end{array}} \mathcal {O}_2(\mathtt P,\mathbf {t}') \quad \hbox {with} \\&\quad \mathcal {O}_2(\mathtt P,\mathbf {t}') {\mathop {=}\limits ^{\text {def}}}\left\| \mathtt P \mathtt X ' + \mathbf {t}' \mathbf {1}^\top - \mathtt Y \right\| _\mathcal {F}^2. \end{aligned}$$

Taking the partial derivatives of $\mathcal {O}_2$ with respect to $\mathbf {t}'$, we obtain:

$$\begin{aligned} 2m\mathbf {t}' - 2m\mathbf {y} + 2m\mathtt P \mathbf {x}' =\mathbf {0}, \end{aligned}$$

where $\mathbf {y} = \frac{1}{m}\mathtt Y \mathbf {1}$ is the centre of gravity of the observed points and $\mathbf {x}' = \frac{1}{m}\mathtt X '\mathbf {1}$. Because the new model points in $\mathtt X '$ were obtained by centring the model points in $\mathtt X$, we have $\mathbf {x}' = \mathbf {0}$. We thus obtain:

$$\begin{aligned} \mathbf {t}' =\mathbf {y}. \end{aligned}$$

Defining $\mathtt Y ' {\mathop {=}\limits ^{\text {def}}}\mathtt Y-\mathbf {y}{\mathbf {1}}^\top$ as a new set of observed points by centring the initial ones, we arrive at the translationless formulation:

$$\begin{aligned} \min _{\mathtt P \in \mathbb {M}_*} \mathcal {O}_3(\mathtt P) \quad \hbox {with} \quad \mathcal {O}_3(\mathtt P) {\mathop {=}\limits ^{\text {def}}}\left\| \mathtt P \mathtt X ' - \mathtt Y ' \right\| _\mathcal {F}^2. \end{aligned}$$

Once the rotational part $\mathtt P$ is estimated, the translation is given by:

$$\begin{aligned} \mathbf {t} =\mathbf {y} - \mathtt P \mathbf {x}. \end{aligned}$$

(35)

Analytic Expression of the Cholesky Factor of $\mathtt I +\mathbf {d}\mathbf {d}^\top$

The Cholesky factor $\mathtt V$ defined as $\mathtt I +\mathbf {d}\mathbf {d}^\top = \mathtt V \mathtt V ^\top$ and its inverse are given by:

$$\begin{aligned} \mathtt V&=\begin{bmatrix} \sqrt{1+d_1^2}&0 \\ \frac{d_1d_2}{\sqrt{1+d_1^2}}&\frac{\sqrt{1+d_1^2+d_2^2}}{\sqrt{1+d_1^2}} \end{bmatrix} \quad \text{ and } \quad \\ {\mathtt V}^{-1}&=\begin{bmatrix} \frac{1}{\sqrt{1+d_1^2}}&0 \\ -\frac{d_1d_2}{\sqrt{1+d_1^2}\sqrt{1+d_1^2+d_2^2}}&\frac{\sqrt{1+d_1^2}}{\sqrt{1+d_1^2+d_2^2}} \end{bmatrix}. \end{aligned}$$

Aligning the Projection Direction to the Z-Axis

We need to choose $\mathtt R _\mathbf {d}\in \mathbb {SO}_3$ such that $\mathtt {\Pi }_\mathbf {d}\mathtt R _\mathbf {d} = [ \mathtt H \; \mathbf {0} ]$ for some matrix $\mathtt H \in GL_2(\mathbb {R})$. This means finding a rotation which aligns the projection direction $\mathbf {d}$ with the z-axis $\mathop {\text {stk}}(0,0,1)$. In IPPE [5], we chose the smallest angle rotation, using Rodrigues’s formula. This has three limitations. (i) The formula fails for $\mathbf {d} = \mathbf {0}$ and is unstable for $\Vert \mathbf {d}\Vert _2$ small. (ii) The formula involves trigonometry. (iii) The analytic expression of the required ${\mathtt H}^{-1}$ and $\det ({\mathtt H}^{-1})$ is very involved.

We propose a solution in radicals which addresses the above three limitations. This is a general solution, which has the smallest rotation solution as a special case. The requirement on $\mathtt R _\mathbf {d}$ is rewritten as $\mathtt {\Pi }_\mathbf {d} = [\mathtt H \;\mathbf {0}]\mathtt R _\mathbf {d}^\top$. Multiplying each side of this equation to the right by its transpose leads to $\mathtt {\Pi }_\mathbf {d}\mathtt {\Pi }_\mathbf {d}^\top = \mathtt H \mathtt H ^\top$, or equivalently to $\mathtt I +\mathbf {d}\mathbf {d}^\top = \mathtt H \mathtt H ^\top$. The general solution to this equation has one free degree of freedom and is given by $\mathtt H = \mathtt V \mathtt F$, with $\mathtt F \in \mathbb {O}_2$ an arbitrary orthonormal matrix and $\mathtt V \in GL_2(\mathbb {R})$ a lower triangular matrix representing a Cholesky factor of $\mathtt I +\mathbf {d}\mathbf {d}^\top = \mathtt V \mathtt V ^\top$. The first two rows of $\mathtt R _\mathbf {d}^\top$ are obtained from $\mathtt {\Pi }_\mathbf {d} = \mathtt H \mathtt {\Pi }_\mathbf {0} \mathtt R _\mathbf {d}^\top$ as $\mathtt {\Pi }_\mathbf {0} \mathtt R _\mathbf {d}^\top = \mathtt F ^\top {\mathtt V}^{-1} \mathtt {\Pi }_\mathbf {d}$, and its third row is then obtained as their cross-product. A simple solution is obtained by choosing $\mathtt F = \mathtt I$ as:

$$\begin{aligned} \mathtt R _\mathbf {d}&{\mathop {=}\limits ^{\text {def}}}\begin{bmatrix} 1&-d_1d_2&-d_1 \\ 0&1+d_1^2&-d_2 \\ d_1&d_2&1 \end{bmatrix} \begin{bmatrix} a''\\ {}&a''b''\\&b'' \end{bmatrix} \quad \hbox {with} \\ a''&{\mathop {=}\limits ^{\text {def}}}\frac{1}{\sqrt{1+d_1^2}} \quad \text{ and } \quad b'' {\mathop {=}\limits ^{\text {def}}}\frac{1}{\sqrt{1+d_1^2+d_2^2}}. \end{aligned}$$

We have that ${\mathtt H}^{-1}$ is the leading $(2\times 2)$ sub-matrix of $\mathtt R _\mathbf {d}^\top$, and $\det ({\mathtt H}^{-1})$ is its bottom-right element. We propose an efficient algorithm to compute the Cholesky factor and its inverse, and matrix $\mathtt R _\mathbf {d}$, given in Table 8.

Table 8 Algebraic procedure to align the projection axis to the Z axis

Solution of the Rank-1 Equation of Type 3, $z\mathbf {u}\mathbf {u}^\top + z\mathtt G = \mathtt K$

Proposition 1

(Rank-1 equation, type 3) Let $\mathtt G \in \mathbb {S}$ and $\mathtt K \in \mathbb {S}$ be two known matrices. The following matrix equation defines three constraints on three unknowns in $\mathbf {u} \in \mathbb {R} ^2$ and $z\in \mathbb {R} ^+$:

$$\begin{aligned} z\mathbf {u}\mathbf {u}^\top + z \mathtt G =\mathtt K. \end{aligned}$$

(36)

Equation (36) has always a unique solution for z and one or two solutions for $\mathbf {u}$. Let $\mathtt V \in \mathbb {R} ^{2\times 2}$ be an upper triangular full rank matrix obtained from the Cholesky decomposition $\mathtt G = \mathtt V \mathtt V ^\top$ and set $\mathtt A \leftarrow {\mathtt V}^{-1}\mathtt K {\mathtt V}^{-\top }$. The solution for z is given by $z = \lambda _2(\mathtt A)$. The two solutions for $\mathbf {u}$ are given by $\mathbf {u} = \pm \mathrm {rank}_1\!\!\left( \frac{1}{\lambda _2(\mathtt A)} \mathtt K -\mathtt G \right)$. They both vanish if $\lambda _1(\mathtt A) = \lambda _2(\mathtt A)$.

Proof

Because $\mathtt G \in \mathbb {S}$, we can always compute its Cholesky decomposition and rewrite Eq. (36) as:

$$\begin{aligned} z\mathbf {e} \mathbf {e}^\top + z\mathtt I =\mathtt A, \end{aligned}$$

(37)

with $\mathbf {e} {\mathop {=}\limits ^{\text {def}}}{\mathtt V}^{-1} \mathbf {u}$ and $\mathtt A {\mathop {=}\limits ^{\text {def}}}{\mathtt V}^{-1}\mathtt K {\mathtt V}^{-\top }$, $\mathtt A \in \mathbb {S}$. Since $z\mathbf {e}{\mathbf {e}}^\top = \mathtt A-z\mathtt I$ is rank-1 positive semi-definite, Eq. (37) is equivalent to:

$$\begin{aligned} \lambda _1(\mathtt A - z\mathtt I)\ge & {} 0 \end{aligned}$$

(38)

$$\begin{aligned} \lambda _2(\mathtt A - z\mathtt I)= & {} 0. \end{aligned}$$

(39)

Equation (39) implies $\det (\mathtt A-z\mathtt I)=0$ and so $\exists j \in \{1,2\}$, $z=\lambda _j(\mathtt A)$. Equation (38) and Lemma 2 in Bartoli et al. [2] then imply that z has a single solution given by $z=\lambda _2(\mathtt A)$. Substituting the solution for z into Eq. (36), we obtain:

$$\begin{aligned} \mathbf {u}\mathbf {u}^\top =\frac{1}{\lambda _2(\mathtt A)} \mathtt K -\mathtt G, \end{aligned}$$

and that $\mathbf {u}$ is then obtained by invoking rank-1 factorization. Multiplying this equation by ${\mathtt V}^{-1}$ to the left and ${\mathtt V}^{-\top }$ to the right, we obtain:

$$\begin{aligned} \mathbf {e}\mathbf {e}^\top =\frac{1}{\lambda _2(\mathtt A)} \mathtt A -\mathtt I, \end{aligned}$$

whose right-hand side vanishes for $\lambda _1(\mathtt A)=\lambda _2(\mathtt A)$, leading to $\mathbf {e} = \mathbf {u} = \mathbf {0}$. $\square$

Solution of the Rank-1 Equation of Type 4, $-z\mathbf {u}\mathbf {u}^\top + z\mathtt G = \mathtt K$

Proposition 2

(Rank-1 equation, type 4) Let $\mathtt G \in \mathbb {S}$ and $\mathtt K \in \mathbb {S}$ be two known matrices. The following matrix equation defines three constraints on three unknowns in $\mathbf {u} \in \mathbb {R} ^2$ and $z\in \mathbb {R} ^+$:

$$\begin{aligned} -z\mathbf {u}\mathbf {u}^\top + z \mathtt G =\mathtt K. \end{aligned}$$

(40)

Equation (40) has always a unique solution for z and one or two solutions for $\mathbf {u}$. Let $\mathtt V \in \mathbb {R} ^{2\times 2}$ be an upper triangular full rank matrix obtained from the Cholesky decomposition $\mathtt G = \mathtt V \mathtt V ^\top$ and set $\mathtt A \leftarrow {\mathtt V}^{-1}\mathtt K {\mathtt V}^{-\top }$. The solution for z is given by $z = \lambda _1(\mathtt A)$. The two solutions for $\mathbf {u}$ are given by $\mathbf {u} = \pm \mathrm {rank}_1\!\!\left( \frac{1}{\lambda _1(\mathtt A)} \mathtt K -\mathtt G \right)$. They both vanish if $\lambda _1(\mathtt A) = \lambda _2(\mathtt A)$.

Proof

The proof is very similar to the proof of proposition 1. The conditions are now:

$$\begin{aligned} \lambda _1(\mathtt A - z\mathtt I)= & {} 0 \end{aligned}$$

(41)

$$\begin{aligned} \lambda _2(\mathtt A - z\mathtt I)\le & {} 0, \end{aligned}$$

(42)

from which we obtain $z=\lambda _1(\mathtt A)$, and allow us to use rank-1 factorization to retrieve $\mathbf {u}$. $\square$

Intersecting a Centred Ellipse and the Unit Circle

1.1 Problem Statement

Given $\bar{\mathtt E} \in \mathbb {R} ^{2\times 2}$, $\bar{\mathtt E} \succeq 0$, representing the centred ellipse, the problem is to solve for the intersection points with coordinates $\mathbf {q}\in \mathbb {R} ^2$ such that:

The intersection points of two conics are generally found in two steps. The first step finds a degenerate member of the pencil defined by the two conics. This degenerate member is a pair of lines, and the second step is to recover and intersect these with one of the conics to retrieve the intersection points.

1.2 Finding the Degenerate Conic as a Pair of Lines

By expanding and equating Eqs. (43) and (44), we have:

$$\begin{aligned} \mathbf {q}^\top \bar{\mathtt E} \mathbf {q} =\mathbf {q}^\top \mathbf {q}, \end{aligned}$$

and thus:

$$\begin{aligned} \mathbf {q}^\top \bar{\mathtt P} \mathbf {q} =0 \quad \hbox {with} \quad \bar{\mathtt P} {\mathop {=}\limits ^{\text {def}}}\bar{\mathtt E}-\mathtt I, \end{aligned}$$

which represents one of the sought degenerate conics in the pencil. The degenerate conic $\mathtt P {\mathop {=}\limits ^{\text {def}}}\mathop {\text {diag}}(\bar{\mathtt P},0)$ generally represents a pair of lines, but degenerates to a single line if $\bar{\mathtt P}$ is rank-1. These two lines intersect at the origin, as $\ker (\mathtt P) = \mathop {\text {stk}}(0,0,1)$. The two lines have homogeneous coefficient vectors $\mathbf {m},\mathbf {l}\in \mathbb {R} ^3$, with $m_3=l_3=0$, as they contain the origin. We have:

$$\begin{aligned} \bar{\mathtt P} {\mathop {=}\limits ^{\text {def}}}\begin{bmatrix} a'&c' \\ c'&b' \end{bmatrix} =\bar{\mathbf {m}}\bar{\mathbf {l}}^\top + \bar{\mathbf {l}}\bar{\mathbf {m}}^\top , \end{aligned}$$

which we expand to:

$$\begin{aligned} a'= & {} 2m_1l_1 \\ b'= & {} 2m_2l_2 \\ c'= & {} m_1l_2+m_2l_1. \end{aligned}$$

Because the role of $\mathbf {m}$ and $\mathbf {l}$ is perfectly similar, we can find both by solving for only one of them. We chose to eliminate $\bar{\mathbf {l}}$ and rewrite the first two equations as $l_1=\frac{a'}{2m_1}$ and $l_2=\frac{b'}{2m_2}$. Substituting into the third equation, we have:

$$\begin{aligned} m_1^2b'+m_2^2a'-2m_1m_2c' =0. \end{aligned}$$

This is a homogeneous quadratic in two variables. We solve for $m_2$ by fixing $m_1=1$ and rescale the result for convenience. Of the two solutions of the quadratic, one is associated with $\bar{\mathbf {m}}$ and one to $\bar{\mathbf {l}}$, giving:

$$\begin{aligned} \bar{\mathbf {m}}&=\frac{\mathop {\mathrm {sign}}(a')}{\sqrt{2|a'|}} \left( \begin{bmatrix} a' \\ c' \end{bmatrix} + \begin{bmatrix} 0 \\ \sqrt{-\det (\bar{\mathtt P})} \end{bmatrix} \right) \quad \text{ and } \quad \\ \bar{\mathbf {l}}&=\frac{1}{\sqrt{2|a'|}} \left( \begin{bmatrix} a' \\ c' \end{bmatrix} - \begin{bmatrix} 0 \\ \sqrt{-\det (\bar{\mathtt P})} \end{bmatrix} \right) . \end{aligned}$$

The sign of $\det (\bar{\mathtt P})$ gives the number of real intersection points up to symmetry about the origin: $\det (\bar{\mathtt P})<0$ corresponds to two real intersection points, $\det (\bar{\mathtt P})=0$ corresponds to one real intersection point, and $\det (\bar{\mathtt P})>0$ corresponds to two complex intersection points.

1.3 Intersecting the Pair of Lines with the Unit Circle

By intersecting the two lines with the unit circle, we retrieve the two solution points up to symmetry, with coordinates $\mathbf {q}_1,\mathbf {q}_2 \in \mathbb {R} ^2$. We have:

$$\begin{aligned} {\bar{\mathbf {m}}}^\top \mathbf {q}_1 =0 \quad \text{ and } \quad \Vert \mathbf {q}_1 \Vert _2 =1, \end{aligned}$$

and:

$$\begin{aligned} {\bar{\mathbf {l}}}^\top \mathbf {q}_2 =0 \quad \text{ and } \quad \Vert \mathbf {q}_2 \Vert _2 =1. \end{aligned}$$

Because we only need $\mathbf {q}_1$ and $\mathbf {q}_2$ up to scale, we have:

$$\begin{aligned} \mathbf {q}_1 \propto \mathtt S \bar{\mathbf {m}} \propto \mathtt S \left( \begin{bmatrix} a'\\ c' \end{bmatrix} + \begin{bmatrix} 0 \\ \sqrt{-\det (\bar{\mathtt P})} \end{bmatrix} \right) , \end{aligned}$$

and:

$$\begin{aligned} \mathbf {q}_2 \propto \mathtt S \bar{\mathbf {l}} \propto \mathtt S \left( \begin{bmatrix} a'\\ c' \end{bmatrix} - \begin{bmatrix} 0 \\ \sqrt{-\det (\bar{\mathtt P})} \end{bmatrix} \right) . \end{aligned}$$

Expanding, and using $\det (\bar{\mathtt P}) = 1+\det (\bar{\mathtt E}) - \text {tr}(\bar{\mathtt E})$, we rewrite these compactly in terms of $\bar{\mathtt E}$ as:

$$\begin{aligned} \mathbf {q}_1\propto & {} \left( \mathtt S (\bar{\mathtt E}-\mathtt I)-\sqrt{\text {tr}(\bar{\mathtt E})-\det (\bar{\mathtt E})-1}\mathtt I \right) \mathbf {e}_1 \\ \mathbf {q}_2\propto & {} \left( \mathtt S (\bar{\mathtt E}-\mathtt I)+\sqrt{\text {tr}(\bar{\mathtt E})-\det (\bar{\mathtt E})-1}\mathtt I \right) \mathbf {e}_1. \end{aligned}$$

Impossibility of Cases $1.\mathtt x.\mathtt x$ Other Than 1.2.2

Case 1.2.2 is studied in Sect. 4.5. We show that all other sub-cases of $1.\mathtt x.\mathtt x$ are impossible under the problem’s inputs.

1.1 Case 1.1.2

We first specialize the parameterization of cases $1.\mathtt x.\mathtt x$ and reduce it to two unknowns. We then show that the orthonormality constraint yields a pair of incompatible equations on one of these two unknowns.

1.1.1 Specializing the Parameterization

We have $\mathrm {rank}(\mathtt M)=\mathrm {rank}(\mathtt M _1)=1$. This is equivalent to $\beta = -\sigma _1^2$, as shown in Sect. 4.5.1. Using parameterization (20), we thus have:

$$\begin{aligned} \mathtt M =- \sigma _1^2 \mathbf {q}\mathbf {q}^\top . \end{aligned}$$

1.1.2 Reducing the Unknowns to $\alpha ,\nu \in \mathbb {R}$

Solving for $\mathbf {q}$. We have from Eq. (16) that:

$$\begin{aligned} \mathtt M _1 =- \sigma _1^2 \mathbf {q}\mathbf {q}^\top + \sigma _1^2\mathtt I =\sigma _1^2 \left( \mathtt I- \mathbf {q}\mathbf {q}^\top \right) . \end{aligned}$$

It is easy to verify that this matrix has $\mathbf {q}$ as nullvector and $\mathtt S \mathbf {q}$ as eigenvector, with eigenvalue $\sigma _1^2$. Still using Eq. (16), we thus have:

$$\begin{aligned} \sigma _1 \left( \mathtt I- \mathbf {q}\mathbf {q}^\top \right) \mathbf {b}_1 =\mathbf {z}_1'. \end{aligned}$$

(45)

Left-multiplying by $\mathbf {q}^\top$ gives $\mathbf {q}^\top \mathbf {z}_1' = 0$. Because $\Vert \mathbf {q}\Vert _2=1$, we obtain:

$$\begin{aligned} \mathbf {q} =\pm \frac{\mathtt S \mathbf {z}_1'}{\Vert \mathbf {z}_1' \Vert _2}. \end{aligned}$$

(46)

Solving for $\mathbf {b}_2$. We have from Eq. (17) that:

$$\begin{aligned} \mathbf {b}_2 =\sigma _2 {\mathtt M}^{-1}_2 \mathbf {z}_2'. \end{aligned}$$

By substituting $\mathtt M _2 = \mathtt M + \sigma _2^2\mathtt I = -\sigma _1^2\mathbf {q}\mathbf {q}^\top + \sigma _2^2\mathtt I$, and $\mathbf {q}$ from Eq. (46), we obtain:

$$\begin{aligned} \mathbf {b}_2 =\sigma _2 { \left( \sigma _2^2\mathtt I- \sigma _1^2 \frac{ \mathtt S \mathbf {z}_1' \mathbf {z}_1'^\top \mathtt S}{\Vert \mathbf {z}_1'\Vert _2^2} \right) }^{-1} \mathbf {z}_2'. \end{aligned}$$

(47)

Solving for $\mathbf {b}_1$. We left-multiply Eq. (45) by its eigenvector $\mathbf {q}^\top \mathtt S ^\top$, giving:

$$\begin{aligned} \sigma _1 \mathbf {q}^\top \mathtt S ^\top \mathbf {b}_1 =\mathbf {q}^\top \mathtt S ^\top \mathbf {z}_1'. \end{aligned}$$

Substituting $\mathbf {q}$ from Eq. (46), we obtain:

$$\begin{aligned} \sigma _1 \mathbf {z}_1'^\top \mathtt S ^\top \mathtt S ^\top \mathbf {b}_1 =\mathbf {z}_1'^\top \mathtt S ^\top \mathtt S ^\top \mathbf {z}_1'. \end{aligned}$$

Because $\mathtt S ^\top \mathtt S ^\top = - \mathtt I$, we arrive at:

$$\begin{aligned} \sigma _1 \mathbf {z}_1'^\top \mathbf {b}_1 =\Vert \mathbf {z}_1'\Vert _2^2. \end{aligned}$$

The general solution to this equation has one free parameter $\nu \in \mathbb {R}$ and takes the form:

$$\begin{aligned} \mathbf {b}_1 =\frac{\mathbf {z}_1'}{\sigma _1} + \nu \mathtt S \mathbf {z}_1'. \end{aligned}$$

(48)

Expressing $\mathbf {b}$. We use the expression $\mathbf {b} = \alpha \mathtt S \mathbf {q}$ from parameterization (21), giving, using Eq. (46):

$$\begin{aligned} \mathbf {b} =\alpha \frac{\mathbf {z}_1'}{\Vert \mathbf {z}_1'\Vert _2}. \end{aligned}$$

(49)

1.1.3 Impossibility

We have a reduced expression of $\mathtt A$ in terms of the two unknowns $\alpha ,\nu \in \mathbb {R}$, from expressions (48) and (47) of $\mathbf {b}_1$ and $\mathbf {b}_2$, and (49) of $\mathbf {b}$. We observe that they have a stronger dependency on $\mathbf {z}_1'$ than on $\mathbf {z}_2'$. We thus use the formulation group to fix $Z_{2,1}=0$, giving the following simplified expression for $\mathtt A$:

$$\begin{aligned} \mathtt A =\begin{bmatrix} \frac{Z_{1,1}}{\sigma _1}&\frac{Z_{1,2}}{\sigma _2}&\alpha \mathop {\mathrm {sign}}(Z_{1,1}) \\ \nu Z_{1,1}&\frac{\sigma _2Z_{2,2}}{\sigma _2^2-\sigma _1^2}&0 \end{bmatrix}. \end{aligned}$$

It is then straightforward to see that the orthonormality constraint $\mathtt A \mathtt A ^\top = \mathtt I$ yields three equations, one quadratic in $\alpha$, $\Vert [1\;0]\mathtt A \Vert _2^2=1$, one quadratic in $\nu$, $\Vert [0\;1]\mathtt A \Vert _2^2=1$, and one linear in $\nu$, $[1\;0]\mathtt A \mathtt A ^\top \mathop {\text {stk}}(0,1)=0$. The linear equation in $\nu$ gives:

$$\begin{aligned} \nu _0 =\frac{\sigma _1Z_{1,2}Z_{2,2}}{(\sigma _1^2-\sigma _2^2)Z_{1,1}^2}, \end{aligned}$$

and the quadratic equation gives two complex solutions:

$$\begin{aligned} \nu _\pm =\pm \frac{\sigma _2Z_{2,2}i}{(\sigma _1^2-\sigma _2^2)Z_{1,1}^2}, \end{aligned}$$

with $i^2 = \sqrt{-1}$. The linear and the quadratic equations are thus compatible only if they both vanish, which occurs if $\sigma _2=0$, which contradicts the hypothesis of non-colinearity of the model points, or $Z_{2,2}=0$, which, because $Z_{2,1}=0$, contradicts the hypothesis that $\mathrm {rank}(\mathtt Z)=2$.

1.2 Case 1.2.1

The proof of impossibility is obtained very similarly to case 1.1.2. We have:

$$\begin{aligned} \mathbf {q}= & {} \pm \frac{\mathtt S \mathbf {z}_2'}{\Vert \mathbf {z}_2'\Vert _2} \\ \mathbf {b}_1= & {} \sigma _1 {\mathtt M}^{-1}_1 \mathbf {z}_1' \\ \mathbf {b}_2= & {} \frac{\mathbf {z}_2'}{\sigma _2} + \nu \mathtt S \mathbf {z}_2' \\ \mathbf {b}= & {} \alpha \frac{\mathbf {z}_2'}{\Vert \mathbf {z}_2'\Vert _2}. \end{aligned}$$

We use to formulation group to set $Z_{1,2}=0$, which leads, using the orthonormality constraint, to two incompatible equations in $\nu$.

1.3 Case 1.1.1

We have $\mathrm {rank}(\mathtt M) = \mathrm {rank}(\mathtt M _1)= \mathrm {rank}(\mathtt M _2) = 1$ and $\sigma _1 = \sigma _2$. Following cases 1.1.2 and 1.2.1, we have:

$$\begin{aligned} \mathbf {q} =\pm \frac{\mathtt S \mathbf {z}_1'}{\Vert \mathbf {z}_1'\Vert _2} \quad \text{ and } \quad \mathbf {q} =\pm \frac{\mathtt S \mathbf {z}_2'}{\Vert \mathbf {z}_2'\Vert _2}. \end{aligned}$$

Therefore, we must have $\mathbf {z}_1' \propto \mathbf {z}_2'$, which contradicts the hypothesis that $\mathrm {rank}(\mathtt Z) = 2$.

1.4 Cases $1.0.\mathtt x$

We have $\mathtt M _1=\mathtt 0$, and from Eq. (16) and parameterization (20):

$$\begin{aligned} \beta \mathbf {q}\mathbf {q}^\top + \sigma _1^2\mathtt I =\mathtt 0. \end{aligned}$$

The off-diagonal equation reduces to $\beta q_1 q_2 = 0$, which means that one of these three variables must vanish. However, the diagonal equations are $\beta q_1^2 + \sigma _1^2 = 0$ and $\beta q_2^2 + \sigma _1^2 = 0$. Because $\sigma _1\not =0$, they, respectively, imply $\beta \not =0$, $q_1\not =0$ and $\beta \not =0$, $q_2\not =0$, which contradicts the off-diagonal equation.

1.5 Cases $1.\mathtt x.0$

We have $\mathtt M _2 = \mathtt 0$ and follow cases $1.0.\mathtt x$ to discard this case.

Algebraic Procedure for the Weak-Perspective Camera

We give our algebraic procedure for the weak-perspective camera, obtained as a simplication of the paraperspective camera case, in Table 9.

Table 9 Algebraic procedure solving weak-perspective planar resection

Derivation Details for Case $2.\mathtt x.\mathtt x$

We derive an optimized solution for case $2.\mathtt x.\mathtt x$ using the formulation give in Sect. 4.4.

Solving for $\mathtt B$. We define $\mathbf {z} {\mathop {=}\limits ^{\text {def}}}\text {vect}(\mathtt Z)$, $\mathbf {g} {\mathop {=}\limits ^{\text {def}}}\mathop {\text {stk}}(a,b)$ and:

$$\begin{aligned} \mathtt K {\mathop {=}\limits ^{\text {def}}}\begin{bmatrix} s\sigma _1 \\&s\sigma _1 \\&-\sigma _2 \\ \sigma _2 \end{bmatrix}. \end{aligned}$$

This allows us to rewrite the cost function as:

$$\begin{aligned} \mathcal {O}_7(a,b,s) =\Vert \mathtt K \mathbf {g} - \mathbf {z} \Vert _2^2. \end{aligned}$$

We find the optimal solution for $\mathbf {g}$. We define a Lagrange multiplier $\ell$ to enforce the constraint $a^2+b^2=1$ and the Lagrangian:

$$\begin{aligned} \mathcal L'(a,b,s,\ell ) {\mathop {=}\limits ^{\text {def}}}\mathcal {O}_7(a,b,s) + \ell \left( \Vert \mathbf {g}\Vert _2^2-1\right) . \end{aligned}$$

Differentiating with respect to $\mathbf {g}$ and nullifying, we obtain:

$$\begin{aligned} \frac{\partial \mathcal L'}{\partial \mathbf {g}} =2{\mathtt K}^\top \mathtt K \mathbf {g} - 2{\mathtt K}^\top \mathbf {z} + 2\ell \mathbf {g} =\mathbf {0}. \end{aligned}$$

We obtain $\mathbf {g}$ as a function of $\ell$ and s as:

$$\begin{aligned}&\mathbf {g} ={\left( \mathtt K ^\top \mathtt K + \ell \mathtt I \right) }^{-1} \mathtt K ^\top \mathbf {z} =\frac{1}{\sigma _1^2+\sigma _2^2+\ell } {\mathtt K}^\top \mathbf {z} \quad \hbox {with} \\&{\mathtt K}^\top \mathbf {z} =\begin{bmatrix} s\sigma _1Z_{1,1}+\sigma _2Z_{2,2} \\ s\sigma _1Z_{2,1} - \sigma _2Z_{1,2} \end{bmatrix}. \end{aligned}$$

We determine two possible values for $\ell$ by substituting the solution for $\mathbf {g}$ into the constraint, giving:

$$\begin{aligned} \left\| {\mathtt K}^\top \mathbf {z}\right\| _2^2 =\left( \sigma _1^2+\sigma _2^2+\ell \right) ^2, \end{aligned}$$

which we expand to the following quadratic in $\ell$:

$$\begin{aligned} \ell ^2 + 2\left( \sigma _1^2+\sigma _2^2\right) \ell + \left( \sigma _1^2+\sigma _2^2\right) ^2 - \left\| {\mathtt K}^\top \mathbf {z}\right\| _2^2 =0. \end{aligned}$$

The discriminant is $\delta = 4\left\| {\mathtt K}^\top \mathbf {z}\right\| _2^2 > 0$, and we thus obtain:

$$\begin{aligned} \ell =- \sigma _1^2 - \sigma _2^2 + r \left\| {\mathtt K}^\top \mathbf {z}\right\| _2 \quad \hbox {with} \quad r \in \{-1,1\}. \end{aligned}$$

As expected, we obtain $\mathbf {g}$ with unitary norm as:

$$\begin{aligned} \mathbf {g} =\frac{r{\mathtt K}^\top \mathbf {z}}{\left\| {\mathtt K}^\top \mathbf {z}\right\| _2}. \end{aligned}$$

The final step is to find the optimal solution for s and r. Substituting the solution for $\mathbf {g}$ into the cost function yields:

$$\begin{aligned} \min _{s,r\in \{-1,1\}} \mathcal {O}_8(s,r) \quad \hbox {with} \quad \mathcal {O}_8(s,r) {\mathop {=}\limits ^{\text {def}}}\left\| \frac{r\mathtt K {\mathtt K}^\top \mathbf {z}}{\left\| {\mathtt K}^\top \mathbf {z}\right\| _2} - \mathbf {z}. \right\| _2^2. \end{aligned}$$

Expanding the cost function, we obtain:

$$\begin{aligned} \mathcal {O}_8(s,r) =\frac{\left\| \mathtt K {\mathtt K}^\top \mathbf {z}\right\| _2^2}{\left\| {\mathtt K}^\top \mathbf {z}\right\| _2^2} + \Vert \mathbf {z}\Vert _2^2 - 2 r\frac{{\mathbf {z}}^\top \mathtt K {\mathtt K}^\top \mathbf {z}}{\left\| {\mathtt K}^\top \mathbf {z}\right\| _2}. \end{aligned}$$

The first and second terms do not depend on r. The third term may be rewritten as:

$$\begin{aligned} - 2 r\frac{{\mathbf {z}}^\top \mathtt K {\mathtt K}^\top \mathbf {z}}{\left\| {\mathtt K}^\top \mathbf {z}\right\| _2} =- 2 r \frac{\left\| {\mathtt K}^\top \mathbf {z}\right\| _2^2}{\left\| {\mathtt K}^\top \mathbf {z}\right\| _2} =- 2 r \left\| {\mathtt K}^\top \mathbf {z}\right\| _2, \end{aligned}$$

and is thus minimized for $r=1$. We now solve for s. The problem becomes:

$$\begin{aligned} \min _{s\in \{-1,1\}} \mathcal {O}_8(s,1). \end{aligned}$$

We have $\Vert \mathtt K \mathbf {h}\Vert _2^2 = (\sigma _1^2+\sigma _2^2) \Vert \mathbf {h}\Vert _2^2$, and we can thus expand the cost function as:

$$\begin{aligned} \mathcal {O}_8(s,1) =\sigma _1^2+\sigma _2^2+\Vert \mathbf {z}\Vert _2^2 - 2\left\| {\mathtt K}^\top \mathbf {z}\right\| _2. \end{aligned}$$

Only the last term depends on s. Because of the negative sign, we have to maximize it, and because it is always positive, we can maximize its square. We thus have:

$$\begin{aligned} \max _{s\in \{-1,1\}} \mathcal {O}_9(s) \quad \hbox {with} \quad \mathcal {O}_9(s) {\mathop {=}\limits ^{\text {def}}}\left\| {\mathtt K}^\top \mathbf {z}\right\| _2^2. \end{aligned}$$

The cost function is expanded to:

$$\begin{aligned} \mathcal {O}_9(s) =\sigma _1^2 \Vert \mathbf {z}_1'\Vert _2^2 + \sigma _2^2 \Vert \mathbf {z}_2'\Vert _2^2+2s\sigma _1\sigma _2\det (\mathtt Z). \end{aligned}$$

Only the last term depends on s, and we thus have $s=\mathop {\mathrm {sign}}(\det (\mathtt Z))$.