Geolocalization with aerial image sequence for UAVs

Abstract

The estimation of geolocation for aerial images is significant for tasks like map creating, or automatic navigation for unmanned aerial vehicles (UAVs). We propose a novel geolocalization method for the UAVs using only aerial images and reference road map. The corresponding road maps of the aerial images are firstly merged into a whole mosaic image using our newly-designed aerial image mosaicking algorithm, where the relative homography transformations between road images are firstly estimated using keypoints tracking in RGB aerial images, and then further refined with registration between detected roads. The geolocalization of the aerial mosaic image is then taken as the problem of registering observed roads in the aerial images to the reference road map under the homography transformation. The registration problem is solved with our fast search algorithm based on a novel projective-invariant feature, which consists of two road intersections augmented with their tangents. Experiments demonstrate that the proposed method can localize the aerial image sequence over an area larger than 1000 km$^2$ within a few seconds.

Appendices

7 Appendix A

We provide the deduction to compute the homography matrix that transforms points from the world coordinate to image coordinate.

Let ${\mathbf {p}}_{w}=\left( x_{w}, y_{w}, z_{w}\right) ^{T}$ be one point expressed in the world coordinate, ${\mathbf {K}}$ be the internal camera parameters, and ${\mathbf {R}}, {\mathbf {t}}$ are the rotation matrix and translation vector of the camera respectively. Then the corresponding pixel point ${\mathbf {p}}_{c}$ for ${\mathbf {p}}_{w}$ can be computed as:

$$\begin{aligned} {\mathbf {p}}_{c}={\mathbf {K}}\left( {\mathbf {R}} {\mathbf {p}}_{w}+{\mathbf {t}}\right) \end{aligned}$$

(20)

When the world coordinate is set as that in Fig. 6, we can get $z_{w}=0$.

The rotation matrix ${\mathbf {R}}$ can be expressed in the form of ${\mathbf {R}}=\left[ \begin{array}{ccc}{{\mathbf {r}}_{1}^{T}}&{{\mathbf {r}}_{2}^{T}}&{{\mathbf {r}}_{3}^{T}}\end{array}\right]$, where ${\mathbf {r}}_{i}^{T}, i=1,2,3$, is the ith column of ${\mathbf {R}}$, and then the Eq. 20 becomes

$$\begin{aligned} \mathbf {p}_{c}=\mathbf {K}\left( \left[ \begin{array}{ccc} {\mathbf {r}_{1}^{T}}&{\mathbf {r}_{2}^{T}}&{\mathbf {r}_{3}^{T}}\end{array}\right] \left[ \begin{array} {c}{x_{w}} \\ {y_{w}} \\ {0}\end{array}\right] + \mathbf {t}\right) =\mathbf {K}\left[ \begin{array}{ccc} {\mathbf {r}_{1}^{T}}&{\mathbf {r}_{2}^{T}}&{\mathbf {t}}\end{array}\right] \left[ \begin{array}{c} {x_{w}} \\ {y_{w}} \\ {1}\end{array}\right] \end{aligned}$$

(21)

Writing ${\mathbf {K}}\left[ \begin{array}{ccc}{{\mathbf {r}}_{1}^{T}}&{{\mathbf {r}}_{2}^{T}}&{{\mathbf {t}}}\end{array}\right]$ as ${\mathbf {H}}={\mathbf {K}}\left[ \begin{array}{lll}{{\mathbf {r}}_{1}^{T}}&{{\mathbf {r}}_{2}^{T}}&{{\mathbf {t}}}\end{array}\right]$ yields

$$\begin{aligned} \mathbf {p}_{c}=\mathbf {H}\left[ \begin{array}{c} {x_{w}} \\ {y_{w}} \\ {1}\end{array}\right] \end{aligned}$$

(22)

here $\left( x_{w}, y_{w}, 1\right) ^{T}$ can be treated as the homogeneous coordinate of point in the ground expressed in the world coordinate. So the homography matrix that transforms a point in the ground to the image plane can be computed as:

$$\begin{aligned} {\mathbf {H}}={\mathbf {K}}\left[ \begin{array}{ccc}{{\mathbf {r}}_{1}^{T}}&{{\mathbf {r}}_{2}^{T}}&{{\mathbf {t}}}\end{array}\right] \end{aligned}$$

(23)

8 Appendix B

Here we provide the detailed information of areas where we performed our geolocalization on synthetic aerial image sequences. The distribution of the 20 areas is shown in Fig. 14, and the total surface area, the total length of the roads, the number of the road intersection are presented in Table 1

Fig. 14

The distribution of areas

Table 1 Detailed information of the 20 areas