Shape-From-Template with Curves

Abstract

Shape-from-Template (SfT) is the problem of using a shape template to infer the shape of a deformable object observed in an image. The usual case of SfT is ‘Surface’ SfT, where the shape is a 2D surface embedded in 3D, and the image is a 2D perspective projection. We introduce ‘Curve’ SfT, comprising two new cases of SfT where the shape is a 1D curve. The first new case is when the curve is embedded in 2D and the image a 1D perspective projection. The second new case is when the curve is embedded in 3D and the image a 2D perspective projection. We present a thorough theoretical study of these new cases for isometric deformations, which are a good approximation of ropes, cables and wires. Unlike Surface SfT, we show that Curve SfT is only ever solvable up to discrete ambiguities. We present the necessary and sufficient conditions for solvability with critical point analysis. We further show that unlike Surface SfT, Curve SfT cannot be solved locally using exact non-holonomic Partial Differential Equations. Our main technical contributions are two-fold. First, we give a stable, global reconstruction method that models the problem as a discrete Hidden Markov Model. This can generate all candidate solutions. Second, we give a non-convex refinement method using a novel angle-based deformation parameterization. We present quantitative and qualitative results showing that real curve shaped objects such as a necklace can be successfully reconstructed with Curve SfT.

Appendices

Appendix A

1.1 $\hbox {SfT}^{1 \rightarrow 3 \rightarrow 2}$: Proof of Proposition 3–Critical Point Definition in $\varphi$

Proof

We start by writing $\mathbf {J}_{\bar{\eta }}$ as a function of $\hat{\varphi }$ from Eq. (4):

$$\begin{aligned} \mathbf {J}_{\bar{\eta }} = \frac{\hat{\varphi }_z \mathbf {J}_{\hat{\varphi }} - \hat{\varphi }'_z \hat{\varphi }}{\hat{\varphi }^2_z}. \end{aligned}$$

(44)

We substitute Eq. (44) in Eq. (10), then express $\xi$ as a function of $\hat{\varphi }$ and $\mathbf {J}_{\hat{\varphi }}$:

$$\begin{aligned} \xi =&\; \frac{1}{\Vert \bar{\eta }\Vert ^2} \left( \frac{1}{\hat{\varphi }^4_z} \left( \hat{\varphi }_z \mathbf {J}_{\hat{\varphi }} - \hat{\varphi }'_z \hat{\varphi } \right) ^2 \right. \nonumber \\&\,\left. -\, \frac{1}{\Vert \bar{\eta }\Vert ^2} \frac{1}{\hat{\varphi }^6_z} \left( \hat{\varphi }_z {\mathbf {J}}^{\top }_{\hat{\varphi }} \hat{\varphi } - \hat{\varphi }'_z {\hat{\varphi }}^{\top }\hat{\varphi } \right) \left( \hat{\varphi }_z {\hat{\varphi }}^{\top } \mathbf {J}_{\hat{\varphi }} - \hat{\varphi }'_z {\hat{\varphi }}^{\top }\hat{\varphi } \right) \right) \nonumber \\ =&\; \frac{1}{\Vert \bar{\eta }\Vert ^2} \frac{1}{\hat{\varphi }^4_z} \left( \left( \hat{\varphi }_z \mathbf {J}_{\hat{\varphi }} - \hat{\varphi }'_z \hat{\varphi } \right) ^2\right. \nonumber \\&\,\left. -\, \frac{1}{\Vert \hat{\varphi }\Vert ^2} \left( \hat{\varphi }_z {\mathbf {J}}^{\top }_{\hat{\varphi }} \hat{\varphi } - \hat{\varphi }'_z {\hat{\varphi }}^{\top }\hat{\varphi } \right) \left( \hat{\varphi }_z {\hat{\varphi }}^{\top } \mathbf {J}_{\hat{\varphi }} - \hat{\varphi }'_z {\hat{\varphi }}^{\top }\hat{\varphi } \right) \right) . \end{aligned}$$

(45)

We expand Eq. (45) and simplify:

$$\begin{aligned} \xi =&\; \frac{1}{\Vert \bar{\eta }\Vert ^2} \frac{1}{\hat{\varphi }^4_z} \left( \hat{\varphi }^2_z {\mathbf {J}}^{\top }_{\hat{\varphi }} \mathbf {J}_{\hat{\varphi }} - \frac{\hat{\varphi }^2_z}{\Vert \hat{\varphi }\Vert ^2} {\mathbf {J}}^{\top }_{\hat{\varphi }} \hat{\varphi } {\hat{\varphi }}^{\top } \mathbf {J}_{\hat{\varphi }} \right) \nonumber \\ =&\; \frac{1}{\Vert \bar{\eta }\Vert ^2} \frac{1}{\hat{\varphi }^2_z} \frac{1}{\Vert \hat{\varphi }\Vert ^2} \left( {\hat{\varphi }}^{\top } \hat{\varphi } {\mathbf {J}}^{\top }_{\hat{\varphi }} \mathbf {J}_{\hat{\varphi }} - {\mathbf {J}}^{\top }_{\hat{\varphi }} \hat{\varphi } {\hat{\varphi }}^{\top } \mathbf {J}_{\hat{\varphi }} \right) . \end{aligned}$$

(46)

We use Definition 1 which gives $\hat{\theta }^2(u_c)\xi (u_c) = 1$ if and only if $u_c$ is a critical point. For this, we express $\hat{\theta }^2\xi$ as a function of $\hat{\varphi }$ and $\mathbf {J}_{\hat{\varphi }}$:

$$\begin{aligned} \hat{\theta }^2\xi =&\; \hat{\varphi }^2_z \; \Vert \bar{\eta }\Vert ^2 \frac{1}{\Vert \bar{\eta }\Vert ^2} \frac{1}{\hat{\varphi }^2_z} \frac{1}{\Vert \hat{\varphi }\Vert ^2} \left( {\hat{\varphi }}^{\top } \hat{\varphi } {\mathbf {J}}^{\top }_{\hat{\varphi }} \mathbf {J}_{\hat{\varphi }} - {\mathbf {J}}^{\top }_{\hat{\varphi }} \hat{\varphi } {\hat{\varphi }}^{\top } \mathbf {J}_{\hat{\varphi }} \right) \nonumber \\ =&\; \frac{1}{\Vert \hat{\varphi }\Vert ^2} \left( {\hat{\varphi }}^{\top } \hat{\varphi } {\mathbf {J}}^{\top }_{\hat{\varphi }} \mathbf {J}_{\hat{\varphi }} - {\mathbf {J}}^{\top }_{\hat{\varphi }} \hat{\varphi } {\hat{\varphi }}^{\top } \mathbf {J}_{\hat{\varphi }} \right) . \end{aligned}$$

(47)

We now replace $\hat{\varphi }$ by its three components $\hat{\varphi }_x$, $\hat{\varphi }_y$ and $\hat{\varphi }_z$:

$$\begin{aligned} \hat{\theta }^2\xi= & {} \; \frac{1}{\Vert \hat{\varphi }\Vert ^2} \left( \left( \hat{\varphi }^2_x + \hat{\varphi }^2_y + \hat{\varphi }^2_z \right) \left( \hat{\varphi }'^2_x + \hat{\varphi }'^2_y + \hat{\varphi }'^2_z \right) \right. \nonumber \\&\,\left. -\, \left( \hat{\varphi }_x \hat{\varphi }_x + \hat{\varphi }_y \hat{\varphi }'_y + \hat{\varphi }_z \hat{\varphi }'_z \right) ^2 \right) . \end{aligned}$$

(48)

By expanding Eq. (48) and simplifying, we obtain:

$$\begin{aligned} \hat{\theta }^2\xi =&\; \frac{1}{\Vert \hat{\varphi }\Vert ^2} \left( \left( \hat{\varphi }_x \hat{\varphi }'_y + \hat{\varphi }'_x \hat{\varphi }_y \right) ^2 + \left( \hat{\varphi }_x \hat{\varphi }'_z + \hat{\varphi }'_x \hat{\varphi }_z \right) ^2 \right. \nonumber \\&\left. +\, \left( \hat{\varphi }_y \hat{\varphi }'_z + \hat{\varphi }'_y \hat{\varphi }_z \right) ^2 \right) \nonumber \\ =&\frac{\Vert \hat{\varphi } \times \mathbf {J}_{\hat{\varphi }} \Vert ^2}{\Vert \hat{\varphi }\Vert ^2}. \end{aligned}$$

(49)

We now reintroduce $u_c$ to use Definition 1:

$$\begin{aligned}&( u_c \text { is a critical point} ) \nonumber \\ {}&\quad \Leftrightarrow \Bigg ( \hat{\theta }^2(u_c)\xi (u_c) = \frac{\Vert \hat{\varphi }(u_c) \times \mathbf {J}_{\hat{\varphi }}(u_c) \Vert ^2}{\Vert \hat{\varphi }(u_c)\Vert ^2} = 1 \Bigg ) \nonumber \\&\quad \Leftrightarrow \Big ( {\hat{\varphi }}^{\top }(u_c) \, \, \mathbf {J}_{\hat{\varphi }}(u_c) = 0 \Big ). \end{aligned}$$

(50)

Appendix B

1.1 $\hbox {SfT}^{1 \rightarrow 3 \rightarrow 2}$: Proof of Proposition 4–The Set of Super Critical Points

Proof

We first demonstrate that given a solution $\hat{\varphi }$ and a critical point $u_c$, then $u_c$ is also a critical point of $\varphi _s$. From Definition 1, we have $\hat{\theta }'(u_c)=0$, which gives:

$$\begin{aligned} \hat{\theta }(u_c) = \frac{1}{\sqrt{\xi (u_c)}}. \end{aligned}$$

(51)

From Eqs. (51) and (13), we have $\hat{\theta }(u_c) = \theta _{s}(u_c)$. Therefore the two curves meet at $u_c$. To demonstrate that $u_c$ is also a critical point of $\varphi _s$, we first differentiate Eq. (10) to obtain the following second-order ODE:

$$\begin{aligned} 2\theta '\theta '' + \xi ' \theta ^2 + 2\xi \theta \theta ' = 0. \end{aligned}$$

(52)

Because $\hat{\theta }$ is a solution to the ODE (10), at $u_c$ we have by substituting Eq. (52):

$$\begin{aligned} \xi '(u_c) \hat{\theta }(u_c)^2 = 0. \end{aligned}$$

(53)

We then differentiate Eq. (13) to obtain the following constraint on $\varphi _s$ at $u_c$:

$$\begin{aligned} \xi (u_c) \theta _{s}(u_c) \theta '_{s}(u_c) + \xi '(u_c) \theta _{s}(u_c)^2 = 0. \end{aligned}$$

(54)

We substitute Eq. (53) into Eq. (54) and use $\hat{\theta }(u_c) = \theta _{s}(u_c)$ to obtain:

$$\begin{aligned} \xi (u_c) \theta _{s}(u_c) \theta '_{s}(u_c) = 0. \end{aligned}$$

(55)

Because $\hat{\varphi }$ is a solution of the ODE (10) and $u_c$ a critical point, we have $\xi (u_c) \hat{\theta }(u_c)^2 = 1$, so $\xi (u_c)$ and $\theta _{s}(u_c)$ cannot be null. We then have $\theta '_{s}(u_c)=0$ and thus $u_c$ is also a critical point of $\varphi _s$.

Appendix C

$\hbox {SfT}^{1 \rightarrow 3 \rightarrow 2}$: Proof of Proposition 5–Super Critical Point Identities

Proof

Derivation of the first identity. We derive a necessary and sufficient condition on $\eta$ that is valid at super critical points. We assume $\hat{\varphi }$ is a solution to Eq. (4) with $u_s$ being a super critical point. We first differentiate Eq. (10) to form the following ODE:

$$\begin{aligned} 2\theta '\theta '' + \xi ' \theta ^2 + 2\xi \theta \theta ' = 0. \end{aligned}$$

(56)

We know that $\hat{\theta }=\varepsilon \hat{\varphi }_{y}$ is a solution to Eq. (56), and $\hat{\theta }'(u_s)=0$ from Definition 1. We substitute $\hat{\theta }$ in Eq. (56) and evaluate the result at $u_s$, obtaining the following:

$$\begin{aligned} \xi '(u_s) \hat{\theta }^2(u_s)= 0. \end{aligned}$$

(57)

Derivation of the second identity. We know $\hat{\theta }^2(u_s)\ne 0$, otherwise $\hat{\varphi }$ would pass through the camera’s origin at $u_s$. We also have that $\xi '(u_s)=0$ from the first super critical point identity. The second identity is found by differentiating $\xi$ as defined in Eq. (10). To express $\xi '$ as a function of $\eta$ and its derivatives, we first define two intermediate terms, $A_{\eta }$ and $B_{\eta }$, and express $\xi '$ using $A_{\eta }$, $B_{\eta }$ and their first derivatives:

$$\begin{aligned} A_{\eta }= & {} {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta } - \frac{1}{\varepsilon ^2} B_{\eta } \quad \text {and} \quad B_{\eta } = {\mathbf {J}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {J}_{\eta }, \end{aligned}$$

(58)

$$\begin{aligned} A'_{\eta }= & {} {\mathbf {H}}^{\top }_{\eta } \mathbf {J}_{\eta } + {\mathbf {J}}^{\top }_{\eta } \mathbf {H}_{\eta } - \frac{1}{\varepsilon ^4}\left( \varepsilon ^2 B'_{\eta } - 2 \varepsilon ' \varepsilon B_{\eta } \right) ,\end{aligned}$$

(59)

$$\begin{aligned} B'_{\eta }= & {} {\mathbf {H}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {J}_{\eta } + {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta } {\eta }^{\top } \mathbf {J}_{\eta } + {\mathbf {J}}^{\top }_{\eta } \eta {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta } \nonumber \\&+ {\mathbf {J}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {H}_{\eta }. \end{aligned}$$

(60)

Because $\eta$, $\mathbf {J}_{\eta }$ and $\mathbf {H}_{\eta }$ are ${\mathbb {R}}^2$-vector, ${\mathbf {H}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {J}_{\eta } = {\mathbf {J}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {H}_{\eta }$ and ${\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta } {\eta }^{\top } \mathbf {J}_{\eta } = {\mathbf {J}}^{\top }_{\eta } \eta {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta }$, which simplifies $B'_{\eta }$:

$$\begin{aligned} B'_{\eta } = 2 {\mathbf {J}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {H}_{\eta } + 2 {\mathbf {J}}^{\top }_{\eta } \eta {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta }. \end{aligned}$$

(61)

From Eqs. (58), (59) and (61), we have:

$$\begin{aligned} \xi ' =&\; \frac{1}{\varepsilon ^4}\left( \varepsilon ^2 A'_{\eta } - 2 \varepsilon ' \varepsilon A_{\eta } \right) \nonumber \\ =&\; \frac{1}{\varepsilon ^4}\left( 2 \varepsilon ^2 {\mathbf {J}}^{\top }_{\eta } \mathbf {H}_{\eta } - B'_{\eta } + 2 \frac{\varepsilon '}{\varepsilon } B_{\eta } - 2 \varepsilon ' \varepsilon {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta } + 2\frac{\varepsilon '}{\varepsilon } B_{\eta } \right) \nonumber \\ =&\; \frac{1}{\varepsilon ^4}\left( 2 \varepsilon ^2 {\mathbf {J}}^{\top }_{\eta } \mathbf {H}_{\eta } - 2 {\mathbf {J}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {H}_{\eta } - 2 {\mathbf {J}}^{\top }_{\eta } \eta {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta }\right. \nonumber \\&\,\left. +\, \frac{4}{\varepsilon ^2} {\eta }^{\top } \mathbf {J}_{\eta } {\mathbf {J}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {J}_{\eta } - 2 {\eta }^{\top } \mathbf {J}_{\eta } {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta } \right) . \end{aligned}$$

(62)

Using ${\eta }^{\top } \mathbf {J}_{\eta } = {\mathbf {J}}^{\top }_{\eta } \eta$, we obtain:

$$\begin{aligned} \xi '= & {} \frac{2}{\varepsilon ^6}\left( \varepsilon ^4 {\mathbf {J}}^{\top }_{\eta } \mathbf {H}_{\eta } - {\eta }^{\top } \mathbf {J}_{\eta } \left( \varepsilon ^2 {\eta }^{\top } \mathbf {H}_{\eta } + 2 \varepsilon ^2 {\mathbf {J}}^{\top }_{\eta } \mathbf {J}_{\eta } \right. \right. \nonumber \\&\left. \left. -\, 2 {\mathbf {J}}^{\top }_{\eta } \eta {\eta }^{\top } \mathbf {J}_{\eta } \right) \right) , \end{aligned}$$

(63)

from which we have that $\xi '(u_s)=0$ is equivalent to:

$$\begin{aligned}&\varepsilon ^4(u_s) {\mathbf {J}}^{\top }_{\eta }(u_s) \mathbf {H}_{\eta }(u_s) - \varepsilon ^2(u_s) {\eta }^{\top }(u_s) \mathbf {J}_{\eta }(u_s) {\eta }^{\top }(u_s) \mathbf {H}_{\eta }(u_s) \nonumber \\&- 2 {\eta }^{\top }(u_s) \mathbf {J}_{\eta }(u_s) \left( \varepsilon ^2(u_s) {\mathbf {J}}^{\top }_{\eta }(u_s) \mathbf {J}_{\eta }(u_s) \right. \nonumber \\&\left. -\, {\mathbf {J}}^{\top }_{\eta }(u_s) \eta (u_s) {\eta }^{\top }(u_s) \mathbf {J}_{\eta }(u_s) \right) = 0. \end{aligned}$$

(64)

By substituting $\varepsilon$ and $\varepsilon '$ in terms of $\eta$ and removing factors in Eq. (64), we obtain the following:

$$\begin{aligned}&\Vert \bar{\eta }(u_s)\Vert ^4 {\mathbf {J}}^{\top }_{\eta }(u_s) \mathbf {H}_{\eta }(u_s) \nonumber \\&- \Vert \bar{\eta }(u_s)\Vert ^2 {\eta }^{\top }(u_s) \mathbf {J}_{\eta }(u_s) {\eta }^{\top }(u_s) \mathbf {H}_{\eta }(u_s) \nonumber \\&- 2 {\eta }^{\top }(u_s) \mathbf {J}_{\eta }(u_s) \left( \Vert \bar{\eta }(u_s)\Vert ^2 {\mathbf {J}}^{\top }_{\eta }(u_s) \mathbf {J}_{\eta }(u_s)\right. \nonumber \\&\left. -\, {\mathbf {J}}^{\top }_{\eta }(u_s) \eta (u_s) {\eta }^{\top }(u_s) \mathbf {J}_{\eta }(u_s) \right) = 0. \end{aligned}$$

(65)

This only depends on $\eta$ and its derivatives.

Derivation of the third identity. We use the fact that, at any super critical point $u_s$, $\hat{\varphi }(u_s) = \varphi _{s}(u_s)$ (Definition 2). We then use Proposition 3 which says that the critical points of the super curve $\varphi _{s}$ are the points where the tangent of $\varphi _{s}$ and the line-of-sight are orthogonal.

Appendix D

1.1 $\hbox {SfT}^{1 \rightarrow 2 \rightarrow 1}$: Proof of Proposition 9–Critical Point Definition in $\varphi$

Proof

We start by writing $\eta '$ in function of $\hat{\varphi }$ from Eq. (17):

$$\begin{aligned} \eta ' =\dfrac{\hat{\varphi }'_x \hat{\varphi }_y-\hat{\varphi }_x\hat{\varphi }'_y}{(\hat{\varphi }_y)^2}. \end{aligned}$$

(66)

We substitute Eq. (66) in Eq. (25) and Eq. (24) to express $\xi$ and $\hat{\theta }$:

$$\begin{aligned} \xi =\dfrac{\left( \hat{\varphi }'_x\hat{\varphi }_y - \hat{\varphi }_x\hat{\varphi }'_y\right) ^2}{\left( \hat{\varphi }^2_x + \hat{\varphi }^2_y \right) ^2} \end{aligned}$$

(67)

$$\begin{aligned} \hat{\theta } =\sqrt{\hat{\varphi }^2_x +\hat{\varphi }^2_y}. \end{aligned}$$

(68)

We use Definition 1 which gives $\hat{\theta }^2(u_c)\xi (u_c) = 1$ if and only if $u_c$ is a critical point. For this, we express $\hat{\theta }^2\xi$ as a function of $\hat{\varphi }$ and $\hat{\varphi }'$:

$$\begin{aligned} \nonumber \hat{\theta }^2\xi =&\; \dfrac{\left( \hat{\varphi }'_x\hat{\varphi }_y - \hat{\varphi }_x\hat{\varphi }'_y) \right) ^2}{\hat{\varphi }^2_x + \hat{\varphi }^2_y} \\ =&\; \dfrac{\Vert \hat{\varphi } \times \hat{\varphi }'\Vert ^2}{\left\| \hat{\varphi }\right\| ^2}. \end{aligned}$$

(69)

We now reintroduce $u_c$ to use Definition 1:

$$\begin{aligned}&\nonumber ( u_c \text { is a critical point} ) \\&\quad \Leftrightarrow \Bigg ( \hat{\theta }^2(u_c)\xi (u_c) = \dfrac{\Vert \hat{\varphi }(u_c) \times \hat{\varphi }'(u_c)\Vert ^2}{\left\| \hat{\varphi }(u_c)\right\| ^2} = 1 \Bigg )\nonumber \\&\quad \Leftrightarrow \Big ( {\hat{\varphi }}^{\top }(u_c) \, \, \hat{\varphi }'(u_c) = 0 \Big ). \end{aligned}$$

(70)

Appendix E

$\hbox {SfT}^{1 \rightarrow 2 \rightarrow 1}$: Proof of Proposition 10–Super Critical Point Identites

Proof

We follow the same steps as the proof of Proposition 5 and obtain that $\xi '(u_s)=0$, which is the first characterization. A second one can be found by differentiating the analytical expression of $\xi$ given in Eq. (25):

$$\begin{aligned} \xi '= \dfrac{2\varepsilon ^4\eta '\eta ''-4\varepsilon ^3\varepsilon '\eta '^2}{\varepsilon ^8}, \end{aligned}$$

(71)

from which we have that $\xi '(u_s)=0$ is equivalent to:

$$\begin{aligned} \dfrac{\varepsilon ^3}{\eta '}\left( \varepsilon (u_s)\eta ''(u_s) - 2\varepsilon '(u_s)\eta '(u_s) \right) =0. \end{aligned}$$

(72)

By substitution of $\varepsilon$ and $\varepsilon '$ in terms of $\eta$ and removing factors in Eq. (72) we have the second identity:

$$\begin{aligned} 2\eta (u_s)\eta '^2(u_s)-(1+\eta ^2(u_s))\eta ''(u_s)=0, \end{aligned}$$

(73)

which only depends on $\eta$ and its derivatives.

For the third characterization, we use the fact that, at any super critical point $u_s$, $\hat{\varphi }(u_s) = \varphi _{s}(u_s)$ (Definition 2). We then use Proposition 9 and obtain that the critical points of the super curve $\varphi _{s}$ are the points where the tangent of $\varphi _{s}$ and the line-of-sight are orthogonal.

Appendix F

1.1 Reconstruction Algorithm of Proposed Category (iv) Method for the $\hbox {SfT}^{1 \rightarrow 3 \rightarrow 2}$ Problem

We give here a pseudo-code of our proposed category (iv) method for the $\hbox {SfT}^{1 \rightarrow 3 \rightarrow 2}$ problem, which we present in Sect. 5.2. The pseudo-code for the $\hbox {SfT}^{1 \rightarrow 2 \rightarrow 1}$ problem differs from only one notation, the 2D correspondence in the input image $\mathbf {q}_k$ which becomes a scalar $q_k$ as it is a 1D correspondence.

Appendix G

Hyperparameters for $\hbox {SfT}^{1 \rightarrow 2 \rightarrow 1}$ and $\hbox {SfT}^{1 \rightarrow 3 \rightarrow 2}$ Experiments

See Tables 5, 6.

Table 5 Hyperparameter values for different category methods to solve $\hbox {SfT}^{1 \rightarrow 2 \rightarrow 1}$ for all datasets

Table 6 Hyperparameter values for different category methods to solve $\hbox {SfT}^{1 \rightarrow 3 \rightarrow 2}$ for all datasets