Vision-based adaptive and recursive tracking of unpaved roads

https://doi.org/10.1016/S0167-8655(01)00120-9Get rights and content

Abstract

We present an approach to road recognition based on Kalman filtering and EM theory resulting in a fast recursive filter that can adaptively track unpaved roads. The road model uses linear dynamic equations and an ANN front end detects road boundaries. Kalman filters construct search neighborhoods and adapt the ANN to road conditions. The road model adapts to motion dynamics using EM. Experimental results are presented.

Introduction

One of the key problems in an autonomous land vehicle (ALV) is the recognition of the road. This is a difficult task since the environment influences the road condition, the road follows a varying path, and the vehicle undergoes complex dynamic motion. The requirement of real-time computation adds to the difficulties.

There have been many implementations of ALV vision systems reported in the literature (Crisman and Thorpe, 1993, Turk et al., 1988, Dickmanns and Mysliwetx, 1992, Waxman et al., 1987, Pomerleau and Jochem, 1996, Yu and Sethi, 1995, Gengenbach, 1995). They can be grouped into two types; model-based approaches and neural net approaches. In the model-based method, the road boundaries are fitted to a mathematical model. This method suffers from three primary drawbacks: (1) the difficulty in obtaining and maintaining precise geometrical road models, (2) the complex algorithms required to search for position and match road features, and (3) excessive computation. In the neural net method, geometrical models are discarded in favor of an artificial neural network (ANN) that learns the models and implicitly represents them via its connection weights. However, their primary problem is processing speed.

Many researchers use a small fixed portion of image to reduce computational loads, accepting the danger of discarding useful information. Many parameters that depend on the road type and the surrounding environment are determined heuristically. Also the dynamic nature of the image sequence is not fully analyzed and properly used.

The motivation of this research is to explore the following three mechanisms: efficient partitioning of the image, use of dynamical information in image sequence, and adaptation. An ANN processes the raw image to find the road boundaries. Instead of a fixed sub image, a small dynamic neighborhood, determined by Kalman filtering (Kalman, 1960), near the present boundary is used. The Kalman filters are based on linear dynamic system models of the road corners, and the final output is used to update the ANN, thus providing adaptation to road and environmental conditions. The linear dynamic system models themselves are actively updated using expectation-maximization (EM) (Dempster et al., 1997) providing adaptation to changing vehicle or road dynamics. The combined system is used to track unpaved roads. The use of a simple model facilitates real time processing and the adaption prevents degradation in performance. Previous use of Kalman filters to get the region surrounding boundaries include (Hsu et al., 1997) and (Arata and Yoshiki, 1996). They are model-based lane recognition methods analyzing off-line sequences in non-real time which showed good results for paved roads with continuous and distinct lane markings. Vehicle dynamics are considered, but the parameter updation method is not proposed.

This paper consists of the following major sections. Section 2 describes the overall structure of the tracking filter. Section 3 deals with the tracking filter. Parameter estimation is addressed in Section 4. Experimental results are given in Section 5 and finally conclusions are presented in Section 6.

Section snippets

Road model and tracking system

In this section, we introduce an efficient representation scheme for the important aspects of the road state and thereby define the road tracking problem as an adaptive prediction filter. The overall road tracking system using this model is also presented.

Road prediction

The MLP uses a gate region to speed processing and provide more coherent data. Kalman filter techniques are used to provide predictions that can be used to obtain the gate region. They also provide optimal estimates of the road trapezoid from the observations.

The Kalman filter equations (Kalman, 1960) for the linear system in (1) arex(k|k)=x(k|k−1)+G(k)y(k)−Hx(k|k−1),x(k+1|k)=Fx(k|k).By alternately processing these two recursive equations we obtain a prediction of the next state x(k+1|k) based

Parameter update

The dynamical equations in (1) contain the parameters Φ={F,Q,R}. The EM loop in Fig. 2 is used to update these parameters so that they do not have to be heuristically determined and can adapt to changing dynamics.

Fig. 4 shows the relationship between the Kalman filter and EM loop. The parameters are not expected to vary rapidly so the EM adaptation is only performed every l frames. The complete data Z(l)={X(l),Y(l)} is comprised of the missing data X(l) which is the state information and is

Experimental results

The road tracking algorithm was tested on a variety of unpaved road image sequences. A color CCD camera provided the images and a video display and capture card digitized the images to 160×120 pixels. This data was processed by a Pentium III (800 MHz) based PC. The gate region height was fixed at h=120 pixels throughout and the gate width was initialized at s(0)=20 pixels. Initial training of the ANN and Kalman filter initialization were facilitated by the manual boundary specification for a

Conclusions

This paper has presented a road tracking system designed for unpaved roads. The recursive prediction of road model that makes it possible to process only a small neighborhood of the image without losing important information. Due to this reduced computation, the tracking system can be fully realized with software only at 20 frames/s on a PC. The automatic parameter adaptation using EM removes the requirement to heuristically determine the road model parameters and allows the system to adapt to

References (12)

  • Arata, T., Yoshiki, N., 1996. Model-based lane recognition. In: Proc. Intelligent Vehicles Symp.'96, pp....
  • J. Crisman et al.

    SCARF: A color vision system that tracks roads and intersections

    IEEE J. Robotics Automation

    (1993)
  • A.P. Dempster et al.

    Maximum likelihood from incomplete data via the EM algorithm

    J. Roy. Statist. Soc. Ser. B

    (1997)
  • E.D. Dickmanns et al.

    Recursive 3-D road and relative ego-state recognition

    IEEE Trans. Pattern Anal. Machine Intell.

    (1992)
  • Gengenbach, V., Nagel, H., Struck, G., Kollnig, H., 1995. Model-based recognition of intersections and lane structures....
  • P.J. Green

    On use of the EM algorithm for penalized likelihood estimation

    J. Roy. Statist. Soc. Ser. B

    (1990)
There are more references available in the full text version of this article.

Cited by (20)

View all citing articles on Scopus
View full text