Elsevier

Image and Vision Computing

Volume 19, Issue 3, February 2001, Pages 119-129
Image and Vision Computing

Road sign detection and recognition using matching pursuit method

https://doi.org/10.1016/S0262-8856(00)00050-0Get rights and content

Abstract

This paper describes an automatic road sign recognition system by using matching pursuit (MP) filters. The system consists of two phases. In the detection phase, it finds the relative position of road sign in the original distant image by using a priori knowledge, shape and color information and captures a closer view image. Then it extracts the road sign image from the closer view image by using conventional template-matching. The recognition phase consists of two processes: training and testing. The training process finds a set of best MP filter bases for each road sign. The testing process projects the input unknown road sign to different set of MP filter bases (corresponding to different road signs) to find the best match.

Introduction

Recently, many intelligent vision systems have been developed for traffic automation [1], [2], [3]. They have many applications such as traffic control and analysis, license plate finding and reading, toll collection, automatic route planning and passive navigation. In this paper, we demonstrate a vision system that can recognize and detect road signs in images of cluttered urban streets as well as country roads. With a camera mounted on a vehicle at a height about 1.7 m, our system can be used to provide the driver with relevant information of the road signs on the scene.

The automatic detection and classification of road signs is clearly an emerging research topic in the field of intelligent vehicle. Different techniques [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14] have been proposed for road sign detection from a sequence of monochromatic or color images. In Ref. [5], triangular, octagonal and circular contours, which are likely to represent the boundaries of road signs, are selected among the closed edge-contours. This makes the algorithm strongly dependent on the quality of the edge detection process. In Ref. [10] a more complicated strategy is followed, which uses both color and edges clues. Edges are tested at different levels of resolution by using so-called a Hierarchical Structure Code, which allows passing from the signal space of an image into the space of its symbolic representation. It is assumed that closed edge-contours are available at one of these levels of resolution, and failures happen when the outline of the traffic sign merges with the background.

A scheme for shape recognition, based on uncertainty handling and multiple knowledge sources combination and propagation, has been applied to correctly segment the images [4]. Tree structure model based systems indexed by shape, color and pictogram features have been implemented for the recognition of the detected road signs [8]. Ritter et al. [11] also used color segmentation algorithm to find the ROIs, which serve as hypotheses as potential road sign. Yabuki et al. [13] examined the color distribution of the road signs to construct the color similarity map. They incorporated the color similarity shown on the map into image function of an active net model. A road sign is extracted as if it is wrapped up in an active net. Similarly, we use color segmentation and template matching processes to detect the road sign. Our method is tolerant to noise, as the geometrical analysis of edges with the color information does not require that the extracted edges have good quality.

Piccioli et al. [6] developed a road sign detection and recognition scheme using a single monochromatic image, which also subdivides the process into three stages: (1) extraction of a search region; (2) shape detection; and (3) recognition. They applied the Kalman-filter-based temporal integration of the extracted information for further improvement. Escalera et al. [12] proposed a road sign detection and classification system, which has two main parts. The first one uses the color thresholding to segment the image and shape analysis to detect the signs. The second one uses a neural network to classify the road signs. They used a receptive field neural network with input layer of 32×32 neurons, output layer of nine neurons and four hidden layers. The net was trained to recognize nine road signs. Gravila [14] developed a multi-feature hierarchical algorithm to match N templates with an image using distance transforms. They used the coarse-to-fine search for the translated parameters and grouped the N templates into template hierarchy based on their similarity. This way, they can match the multi-templates simultaneously at the coarse-level matching and then at fine-level matching, they compute the separate distance transform for the features of each sign.

This paper describes a vision system for road sign detection and recognition. The road signs in color images are acquired by a single camera mounted on a moving vehicle. Since for outdoor scene, the illumination conditions can vary considerably, special attention has been devoted to the robustness and flexibility of the system. The system is achieved by two phases: the detection phase and the recognition phase. In detection phase, we use color extraction and template-matching, along with geometrical reasoning based on a priori knowledge to detect the road signs.

The recognition phase is implemented by using robust and flexible MP filters [15]. The MP filters were introduced to represent the signals or images using an over-complete set of bases called MP bases. Phillps [16] modified the MP filters for solving pattern recognition problem. Different from Ref. [16] who used one set of filters for facial recognition, we use different sets of filters for different road signs. In Ref. [16] there are five basis elements for the MP filter which correspond to five different facial features, whereas, we use eight basis elements for each MP filter set. The MP filters are used to decompose a training pattern into a two-dimensional (2D) wavelet expansion. This yields a representation that is explicitly 2D and encodes information locally, unlike template matching that encodes information globally which is easily influenced by the complex environments. In the recognition phase, it finds the coefficient vector as the feature vector of the input road sign image by applying the MP filters. The road sign recognition system has three concerns: (1) how to recognize road signs with different rotation, translation and scale; (2) the resolution and lighting of the road signs can very considerably; and (3) how to present a good discriminative power with a low on-line computational cost.

Section snippets

Road sign detection

Similar to Ref. [6], our road sign detection system consists of three stages (see Fig. 1). In the first stage, a region in the captured image where the road sign is more likely to be found is selected. Here we use either the color information or a priori information (such as the possible location of the road signs) to identify the region. Therefore, the road sign location is limited to certain designated region, called region of interest (ROI). In the second stage, we search the ROI to find the

Road signs recognition

Once the road sign has been identified, a recognition process is applied to interpret the road sign. The recognition scheme has to consider the following three concerns. First, the output of the detection algorithm is an image of road sign with nearly constant orientation, but the unknown factors, such as camera viewing direction and their relative position, will complicate the input image. Therefore, we need to use as many rotated, translated, or scaled templates as possible for

Experimental results

The experiments are tested for 30 triangular road signs and 10 circular road signs (see Fig. 8). To demonstrate the capability of our system, we perform two different experiments. The first one is the road sign detection, the second one is the road sign identification, it includes recognizing the road sign under different viewing and lightening conditions. Our system is tested mostly in the urban streets in the City of Hsin-Chu, Taiwan. The algorithm has been implemented on Pentium II PC 300 MHz

Conclusion

In this paper, we propose a road sign detection and recognition system. In the detection phase, we have used color features effectively to detect the road signs under noisy and complex environment. In the recognition phase, we use the MP filter to recognize the road signs effectively. The purpose of using the MP filters is to produce the image representation. Our approach can be applied for the development of an automatic pilot system.

References (17)

  • G. Piccioli et al.

    Robust method for road sign detection and recognition

    Image and Vision Computing

    (1996)
  • E.D. Dickmanns et al.

    Recursive 3-D road and relative ego-state recognition

    IEEE Transactions on PAMI

    (1992)
  • D. Koller, N. Heonze, H.-H. Nagel, Algorithmic characterization of vehicle trajectory from image sequences by motion...
  • K. Sakurai et al.

    Analysis of a road image as seen from a vehicle

    ICCV

    (1987)
  • B. Besserer, S. Estable, B. Ulmer, Multiple knowledge sources and evidential reasoning for shape recognition,...
  • M. De Saint Blancard

    Road sign recognition: a study of vision-based decision making for road environment recognition

  • R. Janssen, W. Ritter, F. Stein, S, Ott, Hybrid approach for traffic sign recognition, Proceedings of the Intelligent...
  • L. Priese, V. Rehrmann, R. Schian, R. Lakmann, Traffic sign recognition based on color image evaluation, Proceedings of...
There are more references available in the full text version of this article.

Cited by (0)

View full text