Robust non-local stereo matching for outdoor driving images using segment-simple-tree

Highlights

• We propose a non-local stereo matching algorithm for driver assistance systems.

• The disparity characteristics of outdoor driving images are demonstrated by analysis.

• We introduce segment-simple-tree that is more adequate for outdoor driving images than minimum spanning tree.

• Qualitative and quantitative evaluation to existing methods is provided over three datasets.

Abstract

Non-local cost aggregation has recently emerged as a promising approach for stereomatching and has attracted much interest over the past few years. Most non-local algorithms are reportedly better than state-of-the-art local algorithms for high-quality indoor images. However, the accuracy of non-local algorithms is still limited for outdoor images. Computing disparity maps for outdoor images in driver assistance systems is one of the most actively researched topics in the field of stereo vision. In this paper, we present a robust non-local stereo matching algorithm that improves the performance of non-local approaches for outdoor driving images. The proposed algorithm is inspired by the non-local cost aggregation method based on a minimum spanning tree, and it improves the estimation accuracy by introducing an alternate, effective segment-simple-tree that is more adequate for outdoor driving images than the minimum spanning tree. Experimental results showed that the proposed algorithm is superior to the existing local and non-local algorithms, and is comparable to semi-global matching.

Introduction

Binocular stereo matching is one of the most important algorithms in computer vision because it provides computers with the depth perception ability similar to that of humans. The goal is to estimate the disparity maps from two rectified images of the same scene taken from left and right viewpoints. Binocular stereo matching has been extensively studied over the past few decades, and numerous algorithms have been proposed [1], [2], [3]. Nevertheless, it is still an active area of research because new challenges have surfaced and there are a variety of problems that have not yet been solved. Among these challenges, computing the disparity maps for outdoor images in driver assistance systems (DAS) is one of the most actively researched topics in this field [4], [5], [6].

In general, a stereo matching algorithm consists of four steps: cost computation, cost aggregation, disparity optimization, and post-processing. In this paper, we mainly focus on the cost aggregation step since it has the greatest impact on the accuracy of the estimated disparity map. Most existing cost aggregation algorithms define a local 2-D support window for each pixel and perform summing/averaging operations using the information obtained from the pixels inside of that window. State-of-the-art local algorithms include the adaptive support-weight approach [7], geodesic diffusion [8], and fast cost-volume filtering [9]. These were respectively inspired by bilateral filtering [10], anisotropic diffusion [11], and guided image filtering [12], which are the three most well-known edge-aware image filtering methods.

Semi-global matching (SGM) [13] is one of the best performing stereo matching algorithms for outdoor driving images [4]. SGM performs the cost aggregation step using several global paths instead of the local pixel similarities used in the local approach. Even though this is an optimization-based approach that uses dynamic programming for each path, it can still be considered as a cost aggregation step [14].

In the past few years, non-local cost aggregation has recently emerged as a promising approach since the accuracy and the computation are reportedly better and faster than existing local algorithms. The key idea behind its success is that it aggregates the matching cost using the information from all of the pixels in the image rather than only those inside a specific window, as the aforementioned local algorithms do [7], [8], [9]. Cigla et al. [15] proposed an information permeability algorithm with separable successive weighted summation along the horizontal and vertical directions, and the improved spatio-temporal information permeability filtering is presented in [16]. Pham et al. [17] employed a sequence of 1-D filters to reduce the computation and memory costs relative to conventional 2-D filters by applying domain transform [18].

Yang׳s method [19], [20], the most notable non-local algorithm, carries out the cost aggregation on a minimum spanning tree (MST). The MST is constructed using a subset of edges whose sum of edge-weights is minimal, and the matching cost is then aggregated on this tree within two passes, leaf-to-root and root-to-leaf. Mei et al. [21] improved upon this algorithm by incorporating image segmentation [22] in the tree construction procedure. This results in the new tree structure, which is called segment-tree (ST). Although these methods generate accurate disparity maps for indoor images as shown in [19], [20], [21], they fail to do the same for outdoor driving images. Both MST and ST are not adequate for outdoor driving images, as will be demonstrated by analysis in the later sections. In this paper, we present a robust non-local algorithm that can meet this challenge. Specifically, we perform cost aggregation on an alternate, effective segment-simple-tree (SST) that is more suitable for outdoor driving images than MST and ST.

The remainder of this paper is structured as follows. Section 2 presents the non-local cost aggregation on a MST, which is the inspiration for our algorithm. Section 3 presents the proposed algorithm. Section 4 presents the cost computation, disparity optimization and post-processing. Section 5 presents the experimental results that compare our algorithm to those proposed in prior literature. Section 6 provides the conclusions for this paper.