Vision-based adaptive and recursive tracking of unpaved roads
Introduction
One of the key problems in an autonomous land vehicle (ALV) is the recognition of the road. This is a difficult task since the environment influences the road condition, the road follows a varying path, and the vehicle undergoes complex dynamic motion. The requirement of real-time computation adds to the difficulties.
There have been many implementations of ALV vision systems reported in the literature (Crisman and Thorpe, 1993, Turk et al., 1988, Dickmanns and Mysliwetx, 1992, Waxman et al., 1987, Pomerleau and Jochem, 1996, Yu and Sethi, 1995, Gengenbach, 1995). They can be grouped into two types; model-based approaches and neural net approaches. In the model-based method, the road boundaries are fitted to a mathematical model. This method suffers from three primary drawbacks: (1) the difficulty in obtaining and maintaining precise geometrical road models, (2) the complex algorithms required to search for position and match road features, and (3) excessive computation. In the neural net method, geometrical models are discarded in favor of an artificial neural network (ANN) that learns the models and implicitly represents them via its connection weights. However, their primary problem is processing speed.
Many researchers use a small fixed portion of image to reduce computational loads, accepting the danger of discarding useful information. Many parameters that depend on the road type and the surrounding environment are determined heuristically. Also the dynamic nature of the image sequence is not fully analyzed and properly used.
The motivation of this research is to explore the following three mechanisms: efficient partitioning of the image, use of dynamical information in image sequence, and adaptation. An ANN processes the raw image to find the road boundaries. Instead of a fixed sub image, a small dynamic neighborhood, determined by Kalman filtering (Kalman, 1960), near the present boundary is used. The Kalman filters are based on linear dynamic system models of the road corners, and the final output is used to update the ANN, thus providing adaptation to road and environmental conditions. The linear dynamic system models themselves are actively updated using expectation-maximization (EM) (Dempster et al., 1997) providing adaptation to changing vehicle or road dynamics. The combined system is used to track unpaved roads. The use of a simple model facilitates real time processing and the adaption prevents degradation in performance. Previous use of Kalman filters to get the region surrounding boundaries include (Hsu et al., 1997) and (Arata and Yoshiki, 1996). They are model-based lane recognition methods analyzing off-line sequences in non-real time which showed good results for paved roads with continuous and distinct lane markings. Vehicle dynamics are considered, but the parameter updation method is not proposed.
This paper consists of the following major sections. Section 2 describes the overall structure of the tracking filter. Section 3 deals with the tracking filter. Parameter estimation is addressed in Section 4. Experimental results are given in Section 5 and finally conclusions are presented in Section 6.
Section snippets
Road model and tracking system
In this section, we introduce an efficient representation scheme for the important aspects of the road state and thereby define the road tracking problem as an adaptive prediction filter. The overall road tracking system using this model is also presented.
Road prediction
The MLP uses a gate region to speed processing and provide more coherent data. Kalman filter techniques are used to provide predictions that can be used to obtain the gate region. They also provide optimal estimates of the road trapezoid from the observations.
The Kalman filter equations (Kalman, 1960) for the linear system in (1) areBy alternately processing these two recursive equations we obtain a prediction of the next state based
Parameter update
The dynamical equations in (1) contain the parameters . The EM loop in Fig. 2 is used to update these parameters so that they do not have to be heuristically determined and can adapt to changing dynamics.
Fig. 4 shows the relationship between the Kalman filter and EM loop. The parameters are not expected to vary rapidly so the EM adaptation is only performed every l frames. The complete data is comprised of the missing data which is the state information and is
Experimental results
The road tracking algorithm was tested on a variety of unpaved road image sequences. A color CCD camera provided the images and a video display and capture card digitized the images to 160×120 pixels. This data was processed by a Pentium III (800 MHz) based PC. The gate region height was fixed at h=120 pixels throughout and the gate width was initialized at s(0)=20 pixels. Initial training of the ANN and Kalman filter initialization were facilitated by the manual boundary specification for a
Conclusions
This paper has presented a road tracking system designed for unpaved roads. The recursive prediction of road model that makes it possible to process only a small neighborhood of the image without losing important information. Due to this reduced computation, the tracking system can be fully realized with software only at 20 frames/s on a PC. The automatic parameter adaptation using EM removes the requirement to heuristically determine the road model parameters and allows the system to adapt to
References (12)
- Arata, T., Yoshiki, N., 1996. Model-based lane recognition. In: Proc. Intelligent Vehicles Symp.'96, pp....
- et al.
SCARF: A color vision system that tracks roads and intersections
IEEE J. Robotics Automation
(1993) - et al.
Maximum likelihood from incomplete data via the EM algorithm
J. Roy. Statist. Soc. Ser. B
(1997) - et al.
Recursive 3-D road and relative ego-state recognition
IEEE Trans. Pattern Anal. Machine Intell.
(1992) - Gengenbach, V., Nagel, H., Struck, G., Kollnig, H., 1995. Model-based recognition of intersections and lane structures....
On use of the EM algorithm for penalized likelihood estimation
J. Roy. Statist. Soc. Ser. B
(1990)
Cited by (20)
Road lane segmentation using dynamic programming for active safety vehicles
2003, Pattern Recognition LettersAn Active and Contrastive Learning Framework for Fine-Grained Off-Road Semantic Segmentation
2023, IEEE Transactions on Intelligent Transportation SystemsUnstructured Road Detection Method Based on RGB Maximum Two-Dimensional Entropy and Fuzzy Entropy
2022, International Journal of Ambient Computing and IntelligenceCross-modal Fusion-based Prior Correction for Road Detection in Off-road Environments
2022, IEEE International Conference on Intelligent Robots and Systems