GPU-based parallel construction of compact visual hull meshes

Abstract

Building a visual hull model from multiple two-dimensional images provides an effective way of understanding the three-dimensional geometries inherent in the images. In this paper, we present a GPU accelerated algorithm for volumetric visual hull reconstruction that aims to harness the full compute power of the many-core processor. From a set of binary silhouette images with respective camera parameters, our parallel algorithm directly outputs the triangular mesh of the resulting visual hull in the indexed face set format for a compact mesh representation. Unlike previous approaches, the presented method extracts a smooth silhouette contour on the fly from each binary image, which markedly reduces the bumpy artifacts on the visual hull surface due to a simple binary in/out classification. In addition, it applies several optimization techniques that allow an efficient CUDA implementation. We also demonstrate that the compact mesh construction scheme can easily be modified for also producing a time- and space-efficient GPU implementation of the marching cubes algorithm.

Appendix: Proof of perspective-corrected ratios

We want to reveal the relation between the points $\mathbf{p}^{i}_{w}$, $\mathbf{p}^{s}_{w}$, $\mathbf{p}^{o}_{w}$ in the world space and the mapped points $\mathbf{p}^{i}_{s}$, $\mathbf{p}^{s}_{s}$, $\mathbf{p}^{o}_{s}$ in the screen space (see Fig. 9 again). Note that the transformation from the normalized image space to the screen space is an affine transformation because it involves only translation, scaling, and possibly shearing; so is the view transformation that converts points from the world space to the camera space. Since the affine transformations preserve the ratios of distance along a line, it is enough to consider the mapping between the corresponding points $\mathbf{p}^{i}_{c}$, $\mathbf{p}^{s}_{c}$, $\mathbf{p}^{o}_{c}$ in the camera space and $\mathbf{p}^{i}_{n}$, $\mathbf{p}^{s}_{n}$, $\mathbf{p}^{o}_{n}$ in the normalized image space (see Fig. 14).

Fig. 14

Mapping between the camera space and the normalized image space

Assume that $\mathbf{p}^{i}_{c} = (x^{i}_{c} \ y^{i}_{c} \ z^{i}_{c})^{t}$, $\mathbf{p}^{s}_{c} = (x^{s}_{c} \ y^{s}_{c} \ z^{s}_{c})^{t}$, $\mathbf{p}^{o}_{c} = (x^{o}_{c} \ y^{o}_{c} \ z^{o}_{c})^{t}$, where

$$\mathbf{p}^s_c = \alpha\left( \begin{array}{c} x^o_c \\[1mm] y^o_c \\[1mm] z^o_c \end{array} \right) + (1-\alpha) \left( \begin{array}{c} x^i_c \\[1mm] y^i_c \\[1mm] z^i_c \end{array} \right). $$

Similarly, let $\mathbf{p}^{i}_{n} = (x^{i}_{n} \; y^{i}_{n})^{t}$, $\mathbf{p}^{s}_{n}= (x^{s}_{n} \; y^{s}_{n})^{t}$, $\mathbf{p}^{o}_{n}= (x^{o}_{n} \; y^{o}_{n})^{t}$. When $\mathbf{p}^{s}_{c}$ is transformed into the normalized image space by multiplying the perspective transformation matrix, we get

which, via perspective division, leads to

By the same perspective transformation, it becomes that $(x_{n}^{i} \ y_{n}^{i})^{t} = (\frac{f x_{c}^{i}}{z_{c}^{i}} \ \frac{f y_{c}^{i}}{z_{c}^{i}})^{t}$ and $(x_{n}^{o} \ y_{n}^{o})^{t} = (\frac{f x_{c}^{o}}{z_{c}^{o}} \ \frac{fy_{c}^{o}}{z_{c}^{o}})^{t}$. From these, we obtain that

This implies that $t = \frac{\alpha z_{c}^{o}}{\alpha z^{o}_{c} + (1-\alpha )z^{i}_{c}}$, from which we are led to $\alpha= \frac{t z_{c}^{i}}{t z_{c}^{i} + (1-t)z^{o}_{c}}$.  □