Incremental functional maps for accurate and smooth shape correspondence

Abstract

Incorporating multiscale spectral manifold wavelets preservation into the functional map framework for shape correspondence achieves great results in terms of both efficiency and effectiveness. However, fixing the dimension of the spectral embedding strategy in iterations of optimization is troublesome, such as missing high-frequency information when the dimension is small or getting trapped in local minima at a high dimension. In this paper, we present a simple and efficient method for refining correspondences from low frequency to high frequency with a theoretical guarantee. We formulate a strong constraint where the multiscale spectral manifold wavelets should be preserved at each scale correspondingly in the case of the arbitrary dimension of spectral embeddings. To solve the formula, we deduce two relaxed optimization subproblems and propose an incremental alternating iterative algorithm between the spatial and spectral domains via spectral up-sampling and filtering, computing the functional maps and pointwise maps in turn. Our results demonstrate that our method is robust to noisy initialization and scalable with regard to shape resolutions. The deformable shape correspondence benchmark experiments show our method produces more accurate and smoother results than state of the arts.

Appendices

Appendix

Equivalence of two optimization problems

Here we show the optimization problem (10) and (11) are equivalent when adding a regularizer represented as $\mathcal {R} \left( \mathbf {P} \right) = \left\| \left( {\left( \varvec{\Phi }_{\mathcal {N}}^{k} \right) }{\left( \varvec{\Phi }_{\mathcal {N}}^{k} \right) }^{+} - \mathbf {I} \right) \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k} \right\| _{A_\mathcal {N}}^{2}$ into problem (10).

To begin with, we give the result: for any matrix $\mathbf {X}$, $\mathbf {Y}$ and a matrix $\mathbf {B}$ satisfying $\mathbf {B}^\mathrm {T} \mathbf {A} \mathbf {B} = \mathbf {I}$, i.e., $\mathbf {B}^{+} = \mathbf {B}^\mathrm {T} \mathbf {A}$, where $\mathbf {A}$ is a symmetric positive-definite matrix, and let $\left\| \mathbf {X} \right\| _{A}^{2} = trace(\varvec{\mathbf {X}^\mathrm {T} \mathbf {AX}})$, and then, the following equation holds:

$$\begin{aligned} \left\| \mathbf {BX} - \mathbf {Y} \right\| _{F}^{2} + \left\| \left( \mathbf {BB}^{+} - \mathbf {I} \right) \mathbf {Y} \right\| _{A}^{2} = \left\| \mathbf {BX} - \mathbf {Y} \right\| _{A}^{2}. \end{aligned}$$

(14)

In fact, we observe that subtracting and adding the term $\mathbf {BB}^{+}\mathbf {Y}$ in the $\left\| \mathbf {BX} - \mathbf {Y} \right\| _{A}^{2}$, we get

$$\begin{aligned}&\left\| \mathbf {BX} - \mathbf {Y} \right\| _{A}^{2}\\&\quad = \left\| \mathbf {BX} - \mathbf {BB}^{+}\mathbf {Y} + \mathbf {BB}^{+}\mathbf {Y} -\mathbf {Y} \right\| _{A}^{2} \\&\quad = \left\| \mathbf {B} \left( \mathbf {X} - \mathbf {B}^{+} \mathbf {Y}\right) \right\| _{A}^{2} + \left\| \left( \left( \mathbf {BB}^{+} \right) - \mathbf {I} \right) \mathbf {Y} \right\| _{A}^{2} \\&\qquad + 2 trace\left( \left( \mathbf {B} \left( \mathbf {X} - \mathbf {B}^{+} \mathbf {Y} \right) \right) ^\mathrm {T} \mathbf {A} \left( \mathbf {BB}^{+} - \mathbf {I} \right) \mathbf {Y}\right) . \end{aligned}$$

On the one hand, since $\mathbf {B}^\mathrm {T} \mathbf {A} \mathbf {B} = \mathbf {I}$, we have

$$\begin{aligned}&\left\| \mathbf {B} \left( \mathbf {X} - \mathbf {B}^{+} \mathbf {Y}\right) \right\| _{A}^{2}\\&\quad = trace \left( \left( \mathbf {X} - \mathbf {B}^{+} \mathbf {Y} \right) ^\mathrm {T} \mathbf {B}^\mathrm {T} \mathbf {AB} \left( \mathbf {X} - \mathbf {B}^{+} \mathbf {Y} \right) \right) \\&\quad = \left\| \mathbf {X} - \mathbf {B}^{+} \mathbf {Y} \right\| _{F}^{2}. \end{aligned}$$

On the other hand, note that $\mathbf {B}^\mathrm {T}\mathbf {A} \left( \mathbf {BB}^{+} - \mathbf {I} \right) = \mathbf {B}^{+} - \mathbf {B}^{+} = 0$, where $\mathbf {B}^\mathrm {T} \mathbf {A} \mathbf {B} = \mathbf {I}$ and $\mathbf {B}^{+} = \mathbf {B}^\mathrm {T} \mathbf {A}$ are used.

Therefore, Eq. (14) is given. Let $\mathbf {X} = \mathbf {C}^{*}$, $\mathbf {B} = \varvec{\Phi }_{\mathcal {N}}^{k}$, $\mathbf {A} = \mathbf {A}_\mathcal {N}$, and $\mathbf {Y} = \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k}$, we have

$$\begin{aligned} \begin{aligned}&\left\| \varvec{\Phi }_{\mathcal {N}}^{k}\mathbf {C}^{*} - \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k} \right\| _{F}^{2} + \left\| \left( \varvec{\Phi }_{\mathcal {N}}^{k} \left( \varvec{\Phi }_{\mathcal {N}}^{k}\right) ^{+} - \mathbf {I} \right) \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k} \right\| _{A_{\mathcal {N}}}^{2} \\&\quad = \left\| \varvec{\Phi }_{\mathcal {N}}^{k}\mathbf {C}^{*} - \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k} \right\| _{A_{\mathcal {N}}}^{2}. \end{aligned} \end{aligned}$$

As for the issue that

$$\begin{aligned} \begin{aligned}&\mathop {\arg \min }\limits _{\mathbf {P}} \left\| \varvec{\Phi }_{\mathcal {N}}^{k} \mathbf {C}^{*} - \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k}\right\| _{F}^{2} \\&\quad = \mathop {\arg \min }\limits _{\mathbf {P}} \left\| \varvec{\Phi }_{\mathcal {N}}^{k} \mathbf {C}^{*} - \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k}\right\| _{A_\mathcal {N}}^{2}, \end{aligned} \end{aligned}$$

both problems can be solved by finding the closest column between $\varvec{\Phi }_{\mathcal {N}}^{k} \mathbf {C}^{*}$ and $\varvec{\Phi }_{\mathcal {M}}^{k}$ since the matrix $\mathbf {P}$ is encoded by a pointwise map. To improve accuracy and smoothness of correspondences, we select fast Sinkhorn filter algorithm to solve this optimization problem.

Proof of Remark 1

Here we give the proof that (13) is a relaxed solution to problem (12) under certain conditions.

Proof

Indeed, the analytical solution of problem (12) exists by solving a series of linear equations as follows:

$$\begin{aligned} \begin{aligned} {{\mathbf {C}}_{k}}g_{\mathcal {M}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right) {{\left( \varvec{\Phi }_{\mathcal {M}}^{k} \right) }^{+}}=g_{\mathcal {N}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {N}}^{k} \right)&{{\left( \varvec{\Phi }_{\mathcal {N}}^{k} \right) }^{+}}\mathbf {P}^\mathrm {T}, \\&l=0,1,......,L. \end{aligned} \end{aligned}$$

Contrary to Eq. (8), here the point-to-point mapping matrix $\mathbf {P}$ is known, while the functional map matrix $\mathbf {C}_k$ is unknown and enforces no constrain.

Multiplying $g\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right)$ on the left and right sides of the above equations, then

$$\begin{aligned} \begin{aligned} {{\mathbf {C}}_{k}}g_{\mathcal {M}}^{2}{{\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right) }}=g_{\mathcal {N}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {N}}^{k} \right) {{\left( \varvec{\Phi }_{\mathcal {N}}^{k} \right) }^{+}}&\mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k}g_{\mathcal {M}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right) ,\\&l=0,1,......,L. \end{aligned} \end{aligned}$$

Summing the above equations, we obtain

$$\begin{aligned} \sum \limits _{l=0}^{L}{{{\mathbf {C}}_{k}}g_{\mathcal {M}}^{2}{{\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right) }}}&=\sum \limits _{l=0}^{L}{g_{\mathcal {N}}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {N}}^{k} \right) {{\left( \varvec{\Phi }_{\mathcal {N}}^{k} \right) }^{+}} \\&\quad \times \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k}g_{\mathcal {M}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right) . \end{aligned}$$

Taking $\mathbf {C}_{k}$ out of the summation on the left side of the equation, we get

$$\begin{aligned} {{\mathbf {C}}_{k}}\sum \limits _{l=0}^{L}{g_{\mathcal {M}}^{2}{{\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right) }}}&=\sum \limits _{l=0}^{L}{g_{\mathcal {N}}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {N}}^{k} \right) {{\left( \varvec{\Phi }_{\mathcal {N}}^{k} \right) }^{+}} \\&\quad \times \mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k}g_{\mathcal {M}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right) . \end{aligned}$$

According to Eq. (9), this implies the desired result with

$$\begin{aligned} {{\mathbf {C}}_{k}}=\sum \limits _{l=0}^{L}{g_{\mathcal {N}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {N}}^{k} \right) {{\left( \varvec{\Phi }_{\mathcal {N}}^{k} \right) }^{+}}\mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k}g_{\mathcal {M}}\left( {{s}_{l}}\varvec{\Lambda }_{\mathcal {M}}^{k} \right) }. \end{aligned}$$

$\square$

Equivalence of two functional map representations

To derive the equivalence of two functional map representations if the pointwise correspondence from the shape $\mathcal {M}$ to the shape $\mathcal {N}$ is an isometry, where Laplacian eigenvalues of two shapes are same, none of which are repeating, we first give following result: if the diagonal matrices $\mathbf {A}_l, \mathbf {D}_l, l = 0...L$, satisfy $\sum _{l = 0}^L \mathbf {A}_l\mathbf {D}_l = \mathbf {I}$, $\sum _{l = 0}^L \mathbf {A}_l \mathbf {B}\mathbf {D}_l = \mathbf {B}$ holds for any diagonal matrix $\mathbf {B}$. To see that, we observe that if two diagonal matrices are multiplied, the commutative law of multiplication is satisfied, i.e., $\mathbf {B}\mathbf {D}_l = \mathbf {D}_l \mathbf {B}$. From this, it immediately follows that $\sum _{l = 0}^L \mathbf {A}_l \mathbf {B}\mathbf {D}_l = \sum _{l = 0}^L \mathbf {A}_l\mathbf {D}_l \mathbf {B} = \mathbf {B}$.

For one thing, under the condition of isometry, ZoomOut[22] proved the functional map matrix $\mathbf {C}_k$ is both orthonormal and diagonal. For another thing, when Laplacians of two shapes have same eigenvalues, none of which are repeating, we obtain that $g_{\mathcal {N}}\left( {{s}_{l}}\varvec{\Lambda } _{\mathcal {N}}^{k} \right) = g_{\mathcal {M}}\left( {{s}_{l}}\varvec{\Lambda } _{\mathcal {M}}^{k} \right)$, and all of them are diagonal. Thus, we get the equivalence of two functional map representations with $\mathbf {A}_l = g_{\mathcal {N}}\left( {{s}_{l}}\varvec{\Lambda } _{\mathcal {N}}^{k} \right)$, $\mathbf {D}_l = g_{\mathcal {M}}\left( {{s}_{l}}\varvec{\Lambda } _{\mathcal {M}}^{k} \right)$ and $\mathbf {B} = {{\left( \varvec{\Phi }_{\mathcal {N}}^{k} \right) }^{+}}\mathbf {P}^\mathrm {T}\varvec{\Phi }_{\mathcal {M}}^{k}$, which is a general functional map.