Automatic description of complex buildings from multiple images

Abstract

We present an approach for detecting and describing complex buildings with flat or complex rooftops by using multiple, overlapping images of the scene. We find 3-D rooftop boundary hypotheses from the line and junction features of the images by applying consecutive grouping procedures. First, 3-D features are generated by grouping image features over multiple images, and rooftop hypotheses are generated by neighborhood searches on those features. Probabilistic reasoning, level-of-details, and cues from image-derived unedited elevation data are used at various stages to manage the huge search space for rooftop boundary hypotheses. Three-dimensional rooftop hypotheses generated by above procedures are verified with evidence collected from the images and the elevation data. Expandable Bayesian networks are used to combine evidence from multiple images. Finally, overlap and rooftop analyses are performed to find the final building models. Experimental results are shown on complex buildings.

Introduction

Three-dimensional object description is a key task of computer vision. One practical application for the 3-D object description problem is that of building detection and description from aerial images. It can greatly improve the automation of 2-D or 3-D map generation which can be used in various applications including radiowave reachability tests for wireless communications, computer graphics, virtual reality, and mission planning.

The building detection and description has been an active research area [6]. Early systems used a single intensity image, which were effective for simple buildings [9], [11], [14], [15]. In general, multiple aerial images can be obtained with a small extra cost. Thus, most of the recent work in building detection has focused on the stereo or multi-view analysis [1], [2], [3], [7], [17], [18].

There are several challenges for the building detection and description problem.

• Figure–ground separation: we deal with outdoor images, and it is hard to separate building boundaries from other distracting lines such as road boundaries. Moreover, lines and corners of buildings are often broken and missing due to occlusion or other accidental alignments.

• Representation: as in other 3-D object description problems, the model representation takes an important role in the building detection and description problem. When we use simpler representation, such as extrusions of rectangular rooftops [14], [17], the description result will be more robust but the detection rate will be lower¹ because of its limited representational power. On the contrary, when we use a model of extremely high representational power, such as refined polygonal meshes [2], [1], we can describe many more buildings but the result will be less robust and the level of geometric information we obtain will be too poor (we just get polygonal meshes) that the usability of the result will be very limited. It is very important to find a good representation which has a high enough representational power and rich geometric information, where, at the same time, robust and computationally affordable detection and description algorithm is available.

• Information fusion: the types of available information vary depend on the application. In most cases, stereo or multiple images are available. Range data can be generated from stereo analysis but its quality is not good enough to generate building hypotheses directly because many of the building roofs lack sufficient textures for stereo processing. In addition, nearby trees of similar height makes the use of such range data difficult. Sometimes, accurate range data, such as LIDAR, are available (at high cost). There have been efforts to maximize the use of such high-quality data [1], [4] or to increase the quality of the image-based range data with more than 10 images [20]. In this paper, we focus on the use of the image-derived unedited range data. In this case, how to combine information from images and the low-quality range data is very important.

We present an approach for detecting and describing complex buildings by using multiple, overlapping images of the scene. We use low-quality range data, such as from fully automated stereo processing, as additional information. We apply hypothesize and verify paradigm, where lower-level features are grouped (hypothesized) into higher level ones, then filtered (verified) for the purpose of minimizing the computation (otherwise, the computation will be exponential). We present a unique feature grouping approach of lines and junctions where we keep the low-level properties (and uncertainties) to the highest level. For example, a 3-D line feature is not just two end-points synthesized from 2-D line features, but also includes the actual set of 2-D line features (“member” features), and we intensively use the properties of the member features in the higher-level grouping.

To reduce the computation we apply a level-of-detail technique with probabilistic relaxation and introduce various techniques for the efficient filtering, such as information fusion (with range data), probabilistic height estimation, and the use of expandable Bayesian networks [12]. Our approach shows good description results on complex buildings. Our system detects and describes buildings of polygonal boundaries with complex roofs (including superstructures). Such a level of complexity is unprecedented in previous work (among those which do not use high-quality range data).