Continuous Optimization Framework for Depth Sensor Viewpoint Selection

Abstract

Distinguishing differences between areas represented with point cloud data is generally approached by choosing a optimal viewpoint. The most informative view of a scene ultimately enables to have the optimal coverage over distinct points both locally and globally while accounting for the distance to the foci of attention. Measures of surface saliency, related to curvature inconsistency, extenuate differences in shape and are coupled with viewpoint selection approaches. As there is no analytical solution for optimal viewpoint selection, candidate viewpoints are generally discretely sampled and evaluated for information and require (near) exhaustive combinatorial searches. We present a consolidated optimization framework for optimal viewpoint selection with a continuous cost function and analytically derived Jacobian that incorporates view angle, vertex normals and measures of task related surface information relative to viewpoint. We provide a mechanism in the cost function to incorporate sensor attributes such as operating range, field of view and angular resolution. The framework is evaluated as competing favourably with the state-of-the-art approaches to viewpoint selection while significantly reducing the number of viewpoints to be evaluated in the process.

A Appendix

Lemma:

Let $\mathbf {t}$, $\mathbf {p}$ and $\mathbf {z}$ be three column vectors in $\mathrm{I\!R}^n$ and $\mathbf {R}$ is a $n\times n$ matrix. Then for $\mathbf {u}(\mathbf {R},\mathbf {t})=\frac{\left\Vert \mathbf {R}\mathbf {z}\,\varvec{\times }\,(\mathbf {p}\,-\,\mathbf {t})\right\Vert }{\mathbf {R}\mathbf {z}\,\bullet \, (\mathbf {p}\,-\,\mathbf {t})}$, we have:

$$\begin{aligned} \frac{\partial \mathbf {u}}{\partial \mathbf {t}} = \frac{(\mathbf {R}\mathbf {z}\cdot \mathbf {R}\mathbf {z})}{((\mathbf {p}-\mathbf {t})\cdot \mathbf {R}\mathbf {z})^3\mathbf {u}}\Big ((\mathbf {p}-\mathbf {t})\cdot (\mathbf {p}-\mathbf {t})\mathbf {R}\mathbf {z}-((\mathbf {p}-\mathbf {t})\cdot \mathbf {R}\mathbf {z})(\mathbf {p}-\mathbf {t})\Big ) \end{aligned}$$

(25)

$$\begin{aligned} \frac{\partial \mathbf {u}}{\partial \mathbf {R}} = \frac{((\mathbf {p}-\mathbf {t})\cdot (\mathbf {p}-\mathbf {t}))}{(\mathbf {R}\mathbf {z}\cdot (\mathbf {p}-\mathbf {t}))^3\mathbf {u}}\Big ((\mathbf {R}\mathbf {z}\cdot (\mathbf {p}-\mathbf {t}))\mathbf {R}\mathbf {z}-(\mathbf {R}\mathbf {z}\cdot \mathbf {R}\mathbf {z})(\mathbf {p}-\mathbf {t})\Big )\mathbf {z}^\intercal \end{aligned}$$

(26)

Note: To avoid notational confusion, $\mathbf {u}$ denotes the value of function $\mathbf {u}(\mathbf {R},\mathbf {t})$.

Proof

Assume $\mathbf {a}$ and $\mathbf {b}$ are two column vectors in $\mathrm{I\!R}^n$. The auxiliary variable $\pmb {\kappa }$ is defined as a scalar function of $\mathbf {a},\mathbf {b}$ which first vector, $\mathbf {a}$, is a constant,

$$\begin{aligned} \mathbf {\pmb {\kappa }} = \frac{\Vert \mathbf {a}\times \mathbf {b}\Vert ^2}{(\mathbf {a}\cdot \mathbf {b})^2}= \frac{(\mathbf {b}\cdot \mathbf {b})(\mathbf {a}\cdot \mathbf {a})-(\mathbf {b}\cdot \mathbf {a})^2}{(\mathbf {b}\cdot \mathbf {a})^2}= \frac{(\mathbf {b}\cdot \mathbf {b})(\mathbf {a}\cdot \mathbf {a})}{(\mathbf {b}\cdot \mathbf {a})^2} - 1 \end{aligned}$$

(27)

The differential of $\pmb {\kappa }$ is:

$$\begin{aligned} \begin{aligned} \mathrm {d}\pmb {\kappa }&= \frac{2(\mathbf {b}\cdot \mathrm {d}\,\mathbf {b})(\mathbf {a}\cdot \mathbf {a})}{(\mathbf {b}\cdot \mathbf {a})^2} - \frac{2(\mathbf {b}\cdot \mathbf {b})(\mathbf {a}\cdot \mathbf {a})(\mathbf {a}\cdot \mathrm {d}\,\mathbf {b})}{(\mathbf {b}\cdot \mathbf {a})^3}\\&= \frac{2(\mathbf {a}\cdot \mathbf {a})}{(\mathbf {b}\cdot \mathbf {a})^3}\,\Big ((\mathbf {b}\cdot \mathbf {a})\mathbf {b}-(\mathbf {b}\cdot \mathbf {b})\mathbf {a}\Big )\cdot \mathrm {d}\,\mathbf {b}. \end{aligned} \end{aligned}$$

(28)

$$\begin{aligned} \pmb {\kappa } = \mathbf {u}^2 \implies \mathrm {d}\pmb {\kappa }=2\mathbf {u}\,\mathrm {d}\mathbf {u} \end{aligned}$$

(29)

by substituting:

$$\begin{aligned} \mathbf {a} = \mathbf {R}\mathbf {z} \end{aligned}$$

(30)

$$\begin{aligned} \mathbf {b} = (\mathbf {p}-\mathbf {t}) \end{aligned}$$

(31)

$$\begin{aligned} \mathrm {d}\mathbf {b} = -\mathrm {d}\mathbf {t} \end{aligned}$$

(32)

$$\begin{aligned} \mathrm {d}\mathbf {u} = \frac{\mathrm {d}\pmb {\kappa }}{2\mathbf {u}} \,\,= \frac{(\mathbf {a}\cdot \mathbf {a})}{(\mathbf {b}\cdot \mathbf {a})^3\mathbf {u}}\,\Big ((\mathbf {b}\cdot \mathbf {a})\mathbf {b}-(\mathbf {b}\cdot \mathbf {b})\mathbf {a}\Big )\cdot (-\mathrm {d}\mathbf {t}) \end{aligned}$$

(33)

$$\begin{aligned} \frac{\partial \mathbf {u}}{\partial \mathbf {t}} = \frac{(\mathbf {a}\cdot \mathbf {a})}{(\mathbf {b}\cdot \mathbf {a})^3\mathbf {u}}\,\Big ((\mathbf {b}\cdot \mathbf {b})\mathbf {a}-(\mathbf {b}\cdot \mathbf {a})\mathbf {b}\Big ) \end{aligned}$$

(34)

And also if:

$$\begin{aligned} \mathbf {a} = (\mathbf {p}-\mathbf {t}) \end{aligned}$$

(35)

$$\begin{aligned} \mathbf {b} = \mathbf {R}\mathbf {z} \end{aligned}$$

(36)

$$\begin{aligned} \mathrm {d}\mathbf {b}=\mathrm {d}\mathbf {R}\,\mathbf {z} \end{aligned}$$

(37)

$$\begin{aligned} \begin{aligned} \mathrm {d}\mathbf {u}&= \frac{(\mathbf {a}\cdot \mathbf {a})}{(\mathbf {b}\cdot \mathbf {a})^3\mathbf {u}}\,\Big ((\mathbf {b}\cdot \mathbf {a})\mathbf {b}-(\mathbf {b}\cdot \mathbf {b})\mathbf {a}\Big )\cdot \mathrm {d}\mathbf {R}\,\mathbf {z} \\&= \frac{(\mathbf {a}\cdot \mathbf {a})}{(\mathbf {b}\cdot \mathbf {a})^3\mathbf {u}}\,\Big ((\mathbf {b}\cdot \mathbf {a})\mathbf {b}-(\mathbf {b}\cdot \mathbf {b})\mathbf {a}\Big )\mathbf {z}^\intercal \cdot \mathrm {d}\mathbf {R} \end{aligned} \end{aligned}$$

(38)

$$\begin{aligned} \frac{\partial \mathbf {u}}{\partial \mathbf {R}} = \frac{(\mathbf {a}\cdot \mathbf {a})}{(\mathbf {b}\cdot \mathbf {a})^3\mathbf {u}}\,\Big ((\mathbf {b}\cdot \mathbf {a})\mathbf {b}-(\mathbf {b}\cdot \mathbf {b})\mathbf {a}\Big )\mathbf {z}^\intercal \ \end{aligned}$$

(39)