Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth, Illumination and Albedo

Abstract

Shape from shading (SfS) and stereo are two fundamentally different strategies for image-based 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on a matching process that establishes correspondences across images. This difference in approaching the reconstruction problem yields complementary advantages that are worthwhile being combined. So far, however, most “joint” approaches are based on an initial stereo mesh that is subsequently refined using shading information. In this paper we follow a completely different approach. We propose a joint variational method that combines both cues within a single minimisation framework. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detail-preserving anisotropic second-order smoothness term. Moreover, we extend the resulting model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes the approach applicable to objects with non-uniform albedo as well as to scenes with unknown illumination. Experiments for synthetic and real-world images demonstrate the benefits of our combined approach: They not only show that our method is capable of generating very detailed reconstructions, but also that joint approaches are feasible in practice.

Appendices

Appendix A: Pixel-Based Versus Image-Based Coordinates

1.1 Image Coordinates

In the SfS literature the surface is often parametrised in terms of image coordinates and it is typically assumed to lie at negative Z-coordinates; see e.g. Prados and Faugeras (2005). If we denote the image coordinates by $\hat{\mathbf {x}}_0=(\hat{x}_0,\hat{y}_0)^\top$, such a parametrisation is for instance given by Ju et al. (2016); Maurer et al. (2016):

$$\begin{aligned} \mathcal {S} \left( \hat{\mathbf {x}}_0 \right) = \left( \begin{array}{c} \dfrac{z \; \hat{x}_0}{\mathtt {f}_0} \\ \dfrac{z \; \hat{y}_0}{\mathtt {f}_0} \\ - z \end{array} \right) \; . \end{aligned}$$

(58)

The resulting surface normal then reads

$$\begin{aligned} \tilde{\mathbf {n}}(\hat{\mathbf {x}}_0)= & {} \mathcal {S}_{x}(\hat{\mathbf {x}}_0) \times \mathcal {S}_{y}(\hat{\mathbf {x}}_0) = \frac{z}{\mathtt {f}_0^2} \left( \begin{array}{c} \mathtt {f}_0 \; \hat{\nabla } z \\ \big (\hat{\nabla } z^\top \hat{\mathbf {x}}_0\big ) + z \end{array} \right) , \end{aligned}$$

(59)

where $\hat{\nabla }=(\partial _{\hat{x}_0},\partial _{\hat{y}_0})^\top$ denotes the gradient operator in image coordinates. Please note that this surface normal points outwards, i.e. away from the object.

1.2 Pixel Coordinates

In contrast, in our case, the parametrisation of the surface is given in pixel coordinates and the surface is assumed to lie at positive Z-coordinates; see also Graber et al. (2015). The corresponding surface parametrisation then reads

$$\begin{aligned} \mathcal {S} \left( \mathbf {x}_0 \right) = \left( \begin{array}{c} \dfrac{z \; h_{0,x} \; (x_0 - c_{0,x})}{\mathtt {f}_0} \\ \dfrac{z \; h_{0,y} \; (y_0 - c_{0,y})}{\mathtt {f}_0} \\ z \end{array} \right) \; , \end{aligned}$$

(60)

which yields the surface normal

$$\begin{aligned} \tilde{\mathbf {n}}(\mathbf {x}_0)= & {} \mathcal {S}_{x}(\mathbf {x}_0) \times \mathcal {S}_{y}(\mathbf {x}_0)\\ \nonumber= & {} z \; \frac{h_{0,x} \; h_{0,y}}{\mathtt {f}_0^2} \! \left( \begin{array}{c} - \mathtt {f}_0 \; \dfrac{z_x}{h_{0,x}} \\ - \mathtt {f}_0 \; \dfrac{z_y}{h_{0,y}} \\ \nabla z^\top \left( \mathbf {x}_0 \! - \! \mathbf {c}_0\right) + z \end{array} \right) \\ \nonumber= & {} h_{0,x} \; h_{0,y} \; \frac{z}{\mathtt {f}_0^2} \! \left( \begin{array}{c} - \mathtt {f}_0 \; H^{-1}_{0} \; \nabla z \\ \left( H^{-1}_{0} \nabla z \right) ^\top H_{0} \left( \mathbf {x}_0 \! - \! \mathbf {c}_0\right) + z \end{array} \right) , \end{aligned}$$

(61)

where

$$\begin{aligned} H_{0} = \left( \begin{array}{cc} h_{0,x} &{} 0 \\ 0 &{} h_{0,y} \end{array} \right) \end{aligned}$$

(62)

is a diagonal matrix involving the pixel sizes $h_{0,x}$ and $h_{0,y}$. Please note that due to the positive sign in the third component of the parametrisation this normal is now pointing inwards, i.e. into the object. Inverting the direction hence yields the desired outward normal

$$\begin{aligned} - \tilde{\mathbf {n}}(\mathbf {x}_0)= & {} -\mathcal {S}_{x}(\mathbf {x}_0) \times \mathcal {S}_{y}(\mathbf {x}_0)\nonumber \\= & {} h_{0,x} \; h_{0,y} \; \frac{z}{\mathtt {f}_0^2}\nonumber \\&\cdot \,\left( \begin{array}{c} \mathtt {f}_0 \; H^{-1}_{0} \; \nabla z \\ -\left( H^{-1}_{0} \nabla z \right) ^\top H_{0} \left( \mathbf {x}_0 \! - \! \mathbf {c}_0\right) - z \end{array} \right) . \end{aligned}$$

(63)

1.3 Comparison

Let us now compare the surface normals in Eqs. (59) and (63). On the one hand, one can observe a difference in the sign of the third component. Since the sign of the third component of the illumination vector field changes according to the sign of the third component of the surface parametrisation, the result of the inner product between surface normal and light direction in Eq. (11) is the same for both parametrisations. On the other hand, one can rewrite the image coordinates $\hat{\mathbf {x}}_0$ and the associated depth gradient $\hat{\nabla } z$ in terms of pixel coordinates. It is easy to verify that the corresponding transforms are given by $\hat{\mathbf {x}}_0 \! = \! H_{0} \! \left( \mathbf {x}_0 \! - \! \mathbf {c}_0\right)$ and $\hat{\nabla } z \! = \! H_{0}^{-1} \nabla z$, respectively. Finally, ignoring scale—we are only interested in the direction—one can see that the corresponding unit surface normals are equivalent.

Appendix B: Derivation of the Euler–Lagrange Equations

In order to derive the Euler–Lagrange equations (EL 1)–(EL 4) of the differential energy (25) one must compute its first variation, i.e. the Gâteaux differential. Due to the sum rule one may split the derivation into the contributions of the individual terms in Eqs. (35), (37), (38), (39), (40) and (43). While the derivation of most contributions is straightforward (Gelfand and Fomin 2000), the contributions related to the differential SfS data term in Eq. (37) are more complicated due to the fact that it contains non-local contributions. However, in contrast to typical non-local approaches that rely on patch-based strategies, e.g. Boulanger et al. (2011), the non-local contributions in our case originate from the upwind scheme approximation employed in the SfS term.

Fig. 14

Illustration of the rearranging step (a) performed in Eq. (69).

Hence, in the following, we focus on the derivation of those contributions to the Euler–Lagrange equations that are related to the SfS data term. Therefore, we first compute the Gâteaux differential of $D_{\mathsf {sfs}}^k$ at $( dz^k, \mathbf {dl}^k, \mathbf {d}\varvec{\rho }^k )$ in the direction $\mathbf {v} = \left( v_1,\mathbf {v}_2,\mathbf {v}_3\right) ^\top$, as carried out in Eq. (69). While the first steps are straightforward, the rearrangement of the terms in step (a) is slightly more difficult. In order to factor out the directions $v_1$ evaluated at position $\mathbf {x}_0$ we must account for the additional non-local contributions of the neighbouring positions, i.e. the non-local contributions of $\mathbf {x}_0 - \mathbf {h}_x^k$, $\mathbf {x}_0 + \mathbf {h}_x^k$, $\mathbf {x}_0 - \mathbf {h}_y^k$ and $\mathbf {x}_0 + \mathbf {h}_y^k$, as illustrated in Fig. 14. Please note that for reasons of simplicity we neglect the fact that we do not have non-local contributions outside the image domain $\varOmega$, and hence would have a reduced set of neighbours near the boundary. In the last step (b), we can finally identify the coefficients of the directions $v_1$, $\mathbf {v}_2$ and $\mathbf {v}_3$ with the functional derivatives $\frac{\delta D_{\mathsf {sfs}}^k}{\delta dz^k}$, $\frac{\delta D_{\mathsf {sfs}}^k}{\delta \mathbf {dl}^k }$ and $\frac{\delta D_{\mathsf {sfs}}^k}{\delta \mathbf {d}\varvec{\rho }^k }$ which serve as abbreviations.

Since $\delta D_{\mathsf {sfs}}^k$ must vanish at extrema for any direction including $\mathbf {v} = \left( v_1, \mathbf {0}, \mathbf {0}\right) ^\top$, $\mathbf {v} = \left( 0, \mathbf {v}_2, \mathbf {0}\right) ^\top$, and $\mathbf {v} = \left( 0, \mathbf {0}, \mathbf {v}_3\right) ^\top$, we obtain the following necessary conditions, i.e. Euler–Lagrange equations:

$$\begin{aligned} \frac{\delta D_{\mathsf {sfs}}^k}{\delta dz^k} = 0 , \quad \frac{\delta D_{\mathsf {sfs}}^k}{\delta \mathbf {dl}^k } = 0 , \quad \frac{\delta D_{\mathsf {sfs}}^k}{\delta \mathbf {d}\varvec{\rho }^k } = 0 . \end{aligned}$$

(64)

Please note that the other terms of the differential energy give additional contributions to the Euler–Lagrange equations. They can be computed in the same way as before. Hence, finally we obtain:

$$\begin{aligned} 0 = \frac{\delta E^k}{\delta dz^k} =&\bigg ( \frac{\delta D^k_{\mathsf {stereo}}}{\delta dz^k} + \nu \cdot \frac{\delta D^k_{\mathsf {sfs}} }{\delta dz^k} + \alpha _{z} \cdot \frac{\delta R^k_{\mathsf {depth}} }{\delta dz^k} \nonumber \\&+ \alpha _{\mathbf {l}} \cdot \frac{\delta R^k_{\mathsf {illum}} }{\delta dz^k} + \alpha _{\varvec{\rho }} \cdot \frac{\delta R^k_{\mathsf {albedo}}}{\delta dz^k} + \alpha _{\mathsf {inc}} \cdot \frac{\delta R^k_{\mathsf {inc}} }{\delta dz^k}\bigg ) , \end{aligned}$$

(65)

$$\begin{aligned} 0 = \frac{\delta E^k}{\delta \mathbf {dl}^k } =&\bigg ( \frac{\delta D^k_{\mathsf {stereo}}}{\delta \mathbf {dl}^k } + \nu \cdot \frac{\delta D^k_{\mathsf {sfs}} }{\delta \mathbf {dl}^k } + \alpha _{z} \cdot \frac{\delta R^k_{\mathsf {depth}} }{\delta \mathbf {dl}^k } \nonumber \\&+ \alpha _{\mathbf {l}} \cdot \frac{\delta R^k_{\mathsf {illum}} }{\delta \mathbf {dl}^k } + \alpha _{\varvec{\rho }} \cdot \frac{\delta R^k_{\mathsf {albedo}}}{\delta \mathbf {dl}^k } + \alpha _{\mathsf {inc}} \cdot \frac{\delta R^k_{\mathsf {inc}} }{\delta \mathbf {dl}^k }\bigg ) , \end{aligned}$$

(66)

$$\begin{aligned} 0 = \frac{\delta E^k}{\delta \mathbf {d}\varvec{\rho }^k } =&\bigg ( \frac{\delta D^k_{\mathsf {stereo}}}{\delta \mathbf {d}\varvec{\rho }^k} + \nu \cdot \frac{\delta D^k_{\mathsf {sfs}} }{\delta \mathbf {d}\varvec{\rho }^k} + \alpha _{z} \cdot \frac{\delta R^k_{\mathsf {depth}} }{\delta \mathbf {d}\varvec{\rho }^k} \nonumber \\&+ \alpha _{\mathbf {l}} \cdot \frac{\delta R^k_{\mathsf {illum}} }{\delta \mathbf {d}\varvec{\rho }^k} + \alpha _{\varvec{\rho }} \cdot \frac{\delta R^k_{\mathsf {albedo}}}{\delta \mathbf {d}\varvec{\rho }^k} + \alpha _{\mathsf {inc}} \cdot \frac{\delta R^k_{\mathsf {inc}} }{\delta \mathbf {d}\varvec{\rho }^k} \bigg ) , \end{aligned}$$

(67)

$$\begin{aligned} 0 = \frac{\delta E^k}{\delta \mathbf {du}^k } =&\bigg ( \frac{\delta D^k_{\mathsf {stereo}}}{\delta \mathbf {du}^k} + \nu \cdot \frac{\delta D^k_{\mathsf {sfs}} }{\delta \mathbf {du}^k} + \alpha _{z} \cdot \frac{\delta R^k_{\mathsf {depth}} }{\delta \mathbf {du}^k} \nonumber \\&+ \alpha _{\mathbf {l}} \cdot \frac{\delta R^k_{\mathsf {illum}} }{\delta \mathbf {du}^k} + \alpha _{\varvec{\rho }} \cdot \frac{\delta R^k_{\mathsf {albedo}}}{\delta \mathbf {du}^k} + \alpha _{\mathsf {inc}} \cdot \frac{\delta R^k_{\mathsf {inc}} }{\delta \mathbf {du}^k} \bigg ) . \end{aligned}$$

(68)

which then results in the Euler–Lagrange equations (EL 1)–(EL 4).

$$\begin{aligned}&\delta D_{\mathsf {sfs}}^k \left( dz^k, \mathbf {dl}^k, \mathbf {d}\varvec{\rho }^k; v_1,\mathbf {v}_2,\mathbf {v}_3 \right) = \lim \limits _{t \rightarrow 0} \frac{D_{\mathsf {sfs}}^k \left( dz^k + t \cdot v_1, \mathbf {dl}^k + t \cdot \mathbf {v}_2, \mathbf {d}\varvec{\rho }^k + t \cdot \mathbf {v}_3 \right) - D_{\mathsf {sfs}}^k \left( dz^k, \mathbf {dl}^k, \mathbf {d}\varvec{\rho }^k \right) }{t}\\&\quad = \frac{d}{dt} D_{\mathsf {sfs}}^k \left( dz^k + t \cdot v_1, \mathbf {dl}^k + t \cdot \mathbf {v}_2, \mathbf {d}\varvec{\rho }^k + t \cdot \mathbf {v}_3 \right) \biggr |_{t = 0}\\&\quad = 2 \cdot \! \! \int _{\varOmega _{0}} \sum _{c=1}^{3} \Bigg \{ \Bigg [ {\phi }^{k,c}(\mathbf {x}_0) + \sum _{\mathbf {h}^k \in H^k} \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial z^k(\mathbf {x}_0 \! + \! \mathbf {h}^k)} \big ( dz^{k}(\mathbf {x}_0 \! + \! \mathbf {h}^k) + t \cdot v_1(\mathbf {x}_0 \! + \! \mathbf {h}^k) \big ))\\&\qquad +\,\left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \mathbf {l}^k(\mathbf {x}_0)} \right) ^\top \left( \mathbf {dl}^k(\mathbf {x}_0) + t \cdot \mathbf {v}_2(\mathbf {x}_0) \right) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \varvec{\rho }^k(\mathbf {x}_0)}\right) ^\top \left( \mathbf {d}\varvec{\rho }^k(\mathbf {x}_0) + t \cdot \mathbf {v}_3(\mathbf {x}_0) \right) \Bigg ]\\&\qquad \cdot \,\Bigg [ \sum _{\mathbf {n}^k \in H^k} \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial z^k(\mathbf {x}_0 \! + \! \mathbf {n}^k)} \cdot v_1(\mathbf {x}_0 + \mathbf {n}^k) \right) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \mathbf {l}^k(\mathbf {x}_0)} \right) ^\top \mathbf {v}_2(\mathbf {x}_0) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \varvec{\rho }^k(\mathbf {x}_0)}\right) ^\top \mathbf {v}_3(\mathbf {x}_0) \Bigg ] \Bigg \} \, \mathrm {d}\mathbf {x}_0 \Biggr |_{t = 0}\\&\quad = 2 \cdot \! \! \int _{\varOmega _{0}} \sum _{c=1}^{3} \Bigg \{ \Bigg [ {\phi }^{k,c}(\mathbf {x}_0) + \sum _{\mathbf {h}^k \in H^k} \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial z^k(\mathbf {x}_0 \! + \! \mathbf {h}^k)} \; dz^{k}(\mathbf {x}_0 \! + \! \mathbf {h}^k) + \,\left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \mathbf {l}^k(\mathbf {x}_0)} \right) ^\top \mathbf {dl}^k(\mathbf {x}_0) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \varvec{\rho }^k(\mathbf {x}_0)}\right) ^\top \mathbf {d}\varvec{\rho }^k(\mathbf {x}_0) \Bigg ] \\&\qquad \cdot \, \Bigg [ \sum _{\mathbf {n}^k \in H^k} \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial z^k(\mathbf {x}_0 \! + \! \mathbf {n}^k)} \cdot v_1(\mathbf {x}_0 + \mathbf {n}^k) \right) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \mathbf {l}^k(\mathbf {x}_0)} \right) ^\top \mathbf {v}_2(\mathbf {x}_0) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \varvec{\rho }^k(\mathbf {x}_0)}\right) ^\top \mathbf {v}_3(\mathbf {x}_0) \Bigg ] \Bigg \} \, \mathrm {d}\mathbf {x}_0 \end{aligned}$$

$$\begin{aligned}&\quad {\mathop {=}\limits ^{(a)}} 2 \cdot \! \! \int _{\varOmega _{0}} \sum _{c=1}^{3} \sum _{\mathbf {m}^k \in H^k} \Bigg \{ \Bigg [ {\phi }^{k,c}(\mathbf {x}_0 \! + \! \mathbf {m}^k) + \sum _{\mathbf {h}^k \in H^k} \frac{\partial {\phi }^{k,c}(\mathbf {x}_0 \! + \! \mathbf {m}^k)}{\partial z^k(\mathbf {x}_0 \! + \! \mathbf {m}^k \! + \! \mathbf {h}^k)} dz^{k}(\mathbf {x}_0 \! + \! \mathbf {m}^k \! + \! \mathbf {h}^k) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0 \! + \! \mathbf {m}^k)}{\partial \mathbf {l}^k(\mathbf {x}_0 \! + \! \mathbf {m}^k)} \right) ^\top \mathbf {dl}^k(\mathbf {x}_0 \! + \! \mathbf {m}^k)\nonumber \\&\qquad +\, \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0 \! + \! \mathbf {m}^k)}{\partial \varvec{\rho }^k(\mathbf {x}_0 \! + \! \mathbf {m}^k)} \right) ^\top \mathbf {d}\varvec{\rho }^k(\mathbf {x}_0 \! + \! \mathbf {m}^k) \Bigg ] \cdot \,\frac{\partial {\phi }^{k,c}(\mathbf {x}_0 + \mathbf {m}^k)}{\partial z^k(\mathbf {x}_0 )} \Bigg \} \cdot v_1(\mathbf {x}_0) \nonumber \\&\quad + \,\sum _{c=1}^{3} \Bigg \{ \Bigg [ {\phi }^{k,c}(\mathbf {x}_0) + \sum _{\mathbf {h}^k \in H^k} \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial z^k(\mathbf {x}_0 \! + \! \mathbf {h}^k)} \; dz^{k}(\mathbf {x}_0 \! + \! \mathbf {h}^k) +\, \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \mathbf {l}^k(\mathbf {x}_0)} \right) ^\top \mathbf {dl}^k(\mathbf {x}_0) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \varvec{\rho }^k(\mathbf {x}_0)}\right) ^\top \mathbf {d}\varvec{\rho }^k(\mathbf {x}_0) \Bigg ]\nonumber \\&\qquad \cdot \, \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \mathbf {l}^k(\mathbf {x}_0)} \right) \Bigg \}^\top \mathbf {v}_2(\mathbf {x}_0) \nonumber \\&\quad + \,\sum _{c=1}^{3} \Bigg \{ \Bigg [ {\phi }^{k,c}(\mathbf {x}_0) + \sum _{\mathbf {h}^k \in H^k} \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial z^k(\mathbf {x}_0 \! + \! \mathbf {h}^k)} \; dz^{k}(\mathbf {x}_0 \! + \! \mathbf {h}^k) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \mathbf {l}^k(\mathbf {x}_0)} \right) ^\top \mathbf {dl}^k(\mathbf {x}_0) + \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \varvec{\rho }^k(\mathbf {x}_0)}\right) ^\top \mathbf {d}\varvec{\rho }^k(\mathbf {x}_0) \Bigg ] \nonumber \\&\qquad \cdot \, \left( \frac{\partial {\phi }^{k,c}(\mathbf {x}_0)}{\partial \varvec{\rho }^k(\mathbf {x}_0)}\right) \Bigg \}^\top \mathbf {v}_3(\mathbf {x}_0) \; \mathrm {d}\mathbf {x}_0 {\mathop {=}\limits ^{(b)}} 2 \cdot \! \! \int _{\varOmega _{0}} \frac{\delta D_{\mathsf {sfs}}^k}{\delta dz^k}(\mathbf {x}_0) \cdot v_1(\mathbf {x}_0) + \frac{\delta D_{\mathsf {sfs}}^k}{\delta \mathbf {dl}^k }(\mathbf {x}_0) ^\top \mathbf {v}_2(\mathbf {x}_0) + \frac{\delta D_{\mathsf {sfs}}^k}{\delta \mathbf {d}\varvec{\rho }^k }(\mathbf {x}_0) ^\top \mathbf {v}_3(\mathbf {x}_0) \; \mathrm {d}\mathbf {x}_0 \end{aligned}$$

(69)

Appendix C: Pseudocode

Algorithm 1 sketches the employed optimisation strategy in terms of pseudocode.

Appendix D: Model Parameters

In order to obtain the best possible reconstruction the parameters have to be adjusted properly. While the solver related parameters mainly effect the trade off between reconstruction quality and runtime performance and thus can be set fixed in advance (see Sect. 5), the model parameters have direct impact on the quality and thus must be selected more carefully. In this context, a convenient strategy is to first adjust the parameters of the pure stereo model and then extend the parameter set to the parameters of the combined model. From our experience the optimal model parameters may vary depending on the intrinsic camera parameters as well as the distance of the scene to the camera. In this context, even some of the model parameters turned out to be suitable for a wider range of images and thus can also be set fixed in advance; see Sect. 5. What remains to be adjusted are essentially the hyper parameters for the terms of the differential energy, which we specified in Table 2. The parameter settings for the method of Galliani et al. (2015) and Graber et al. (2015) are also given in Table 2. Regarding the method of Graber et al. we even optimised the level depended parameter scaling function in the python code from

$$\begin{aligned} \lambda _{l} = \lambda \cdot \left( \frac{1}{\eta }\right) ^l \quad \text{ to } \quad \lambda _{l} = \lambda \cdot \left( \frac{1}{\eta }\right) ^{1.5 \cdot l} , \end{aligned}$$

(70)

which led to a noticeable improvement of the results. In this context l denotes the current pyramid level (the finest resolution is $l=0$).

Table 2 Parameter settings for the different datasets and approaches