Robust Tramway Detection in Challenging Urban Rail Transit Scenes

Abstract

With the rapid development of light rail transit, tramway detection based on video analysis is becoming the prerequisite and necessary task in driver assistance system. The system should be capable of automatically detecting the trackway using on-board camera in order to determine the train driving limit. However, due to the diversification of ground types, the diversity of weather conditions and the differences in illumination situations, this goal is very challenging. This paper presents a real-time tramway detection method that can effectively deal with various challenging scenarios in the real world of urban rail transit environment. It first uses an adaptive multi-level threshold to segment the ROI of the trolley track, where the local cumulative histogram model is used to estimate the threshold parameters. And then use the regional growth method to reduce the impact of environmental noise and predict the trend of tramway. We have experimentally proved that the method can correctly detect the tramway even in many undesirable situations and use less computational time to meet real-time requirements.

1 Introduction

In recent years, public transport has been greatly advocated due to the desire of alleviating traffic congestion in metropolitan areas and the demand of reducing air emissions that induce climate change. Among various public transport modes, a tram is typically a light-rail public transport vehicle, which is faster than buses and much cheaper than rapid transit systems [9]. The term “tram” is called in Europe and also known as “streetcar” or “trolley-car” in North America. A tram vehicle often runs on tracks along city streets, sometimes on segregated rights-of-way (ROW) in public urban areas [16]. The kind of tram transport have a lot of benefits over bus and other rapid transit trains, such as lower construction cost, improving safety and reliability, better ridership and fewer carbon emissions etc.

Indeed, a tram transport system is technically different from bus systems and other high-capacity rapid rail transit systems. These differences include: (i) the degree of separation of rail tracks, (ii) vehicle technology, and (iii) operating practices [5, 7]. As one of the key features, the degree of separation of rail tracks shows the flexibility of rights-of-way. Most commonly, most of the trams travel in the middle of street strips or at roadsides. On the one hand, although tram routes are mixed with other traffic modes, tram vehicles still keep exclusive or semi-exclusive rights-of-way, which allow them much faster than the buses. On the other hand, the tram is different from a rapid transit line, which runs only on fully grade-separated rights-of-way, such as in tunnels or on elevated structures. A tram transport system often make use of hybrid rights-of-way. A usual design is to permit a tram line with a partial grade-separated rights-of-way and some grade crossings. For example, a tram running on the ground can over-pass or under-pass at heavy-traffic regions or overcrowding intersections and stop only at few street-level crossings. Such a design can reduce significantly construction costs [7, 8].

However, the mode of hybrid rights-of-way is unavoidably bringing potential safety hazards to tram operating practices. Unlike other rapid rail transits, whose rail lines are not accessible and have no interaction with road users, the tram lines in certain areas, such as road crossings, are open and with potential dangers by littering and causing obstructions to the tramway line [15]. In addition, on a street-level crossing, a tram may collide with pedestrians, cyclists and drivers, as it cannot stop so quickly or avoid them. Most commonly, a tram traveling at just 40 kph needs around 50 m to stop. People may not fully realize the length of the braking distance of trams, and so it often ends in tragedy [12, 14]. There are some other dangerous behaviors related to tram operations, such as crossing the tramway line from non-designated areas, entering a restricted area, interfering with the operation of the tramway or taking any action that would compromise the safety of the tram, and crossing the red light at the junction with the tramway and blocking the tramway without permission etc. [16].

Therefore, one of the key technical challenges for modern tram is to survey the track along a tram line with respect to obstacles. The track operated by a tram line is called tramway. Traditionally, a human tram driver uses visual perception to observe the tramway and triggers prompt actions, like whistling and braking, to avoid the occurrence of an accident. Recent studies and accident analysis show that only such the manual survey is unreliable and inefficient [13, 14, 18]. It is mainly due to the following two factors. One is that the nature of a human tram driver’s attention is hard to concentrate for a long time. Another is that human vision is limited under a narrow scale, especially in cases of bad weather or weak illumination. Hence, it is urgent to establish a fully-automated video-based surveillance system for modern tram.

Fig. 1.

A basic vision-based tramway detection model.

Recently, driving assistance technologies in rapid rail transit have been paid so many attentions for improving rail safety. There are various methods and systems designed to help drivers for driving safety, but few publications are published specifically for the detection of tramway using on-board camera. The majority of such the papers been published used threshold segmentation or edge detection to extract the railways. Some methods used the iterative threshold and Otsu method within the simple scenes, where two common edge detection operators: Sobel and Canny were used to extract the edge of track [3, 13]. These methods are difficult to extract the edges correctly in the complex background and are too parameter-sensitive to be estimated. Wohlfeil proposed a set of LLPD operators and used a Hough transform to merge several short linear segments into a continuous line [17]. The method based on Hough transform has an inhibitory effect on the noise and environmental disturbance, and is also applied in the situation when straight line breaks or part of pixels are lost [4, 17]. Espino use a sliding window approach to iteratively select local maxima in the gradient images to extract railway tracks for rubber-tyre trains [2]. Qi computed HOG features, constructed integral images and then extracted railway tracks by region-growing algorithm [10].

Fig. 2.

Different traffic environment scenarios and varying weather and illumination conditions

In fact, the issue of using on-board machine vision to assist train drivers to work and ensure safe operation of train, specially tram, is not easy to address. A basic vision-based railway detection system can be seen in Fig. 1. The core of the technical system in tram is tramway scene analysis. Tramway scene analysis consists of tramway detection and obstacle detection. Tramway detection is an essential and foregoing task, which includes the localization of tram rail tracks and the determination of traffic safety limits. Normally, the video frames acquisited by on-board camera often contain a large number of irrelevant entities, which would cause bad accuracy and low efficiency of obstacle detection. The region of interest (ROI) of obstacle detection is indeed around the tramways and within its surrounding limited area. So accurate tramway detection is extremely important. But a tram running in urban environment often has to be operated in different traffic environment scenarios and under varying weather and illumination conditions, as shown in Fig. 9. At the time, tramway detection becomes non trivial particularly with the presence of various ground types (e.g. Fig. 2(a), (b), (c) and (d)), the effects of weather conditions (e.g. Fig. 2(e), (f) and (g)) and the time of acquisition (e.g. Fig. 2(h), (i) and (j)) and the presence of obstacles occurrence (e.g. Fig. 2(k) and (l)). Its difficulty gets further accentuated in cases of on-board camera movements, blur effects, and abrupt or unexpected actions of drivers, which are likely to distort the result of detection.

Aiming at these challenging situations, this paper proposed a real-time tramway detection method that deals with various challenging situations in real-world urban rail traffic scenarios. By efficiently using a on-board long-distance camera, our perception system figures out the tramway and its corresponding optimal safety limits. Our real-time tramway detection system is distinguished from related researches at the following points.

• (i) Our approach can reliably deal with challenging urban environments, including various ground types, different weather and illumination conditions as well as varying time of acquisition.

• (ii) An adaptive multilevel thresholding method is proposed for segmenting the ROIs of tramway, in which threshold parameters are estimated using local accumulated histogram.

• (iii) The proposed method extracts the optimal tramway in front of the tram vehicle using region growing, instead of recognizing every pixel on every video frame, which is believe to be time-consuming and infeasible for autonomous driving assistance system.

The methods we presented here can work potentially as a cost-effective tram-driving assistance system. We use a simple setup with a surveillance camera plus adaptive algorithm since we don’t want to rely on expensive equipment.

The remainder of the paper is organized as follows. In Sect. 2, we give a detailed introduction of recognition method, including track region segmentation based on multilevel thresholding, adaptive threshold parameters design based on local accumulation histogram, and feature points extraction based on pixel tracing. In Sect. 3, we evaluate our method using various challenging scenarios in real urban traffic environment and analyze some experimental results. Finally, in Sect. 4, we summarize the conclusions and give an outlook on future work.

Fig. 3.

Flangeway and its gray histogram

2 Hybrid Tramway Detection Using Multilevel Thresholding and Region Growing

Tramway detection is mainly divided into two parts, tramway ROI segmentation and tramway feature points extraction. For tramway ROI segmentation, we present a method for ROI segmentation of tramway based on multilevel thresholding and a corresponding algorithm for adaptive threshold parameters using local accumulation histogram. It can extract the grey image of the tramway accurately and effectively. For tramway feature points extraction, we propose a pixel tracing and region growing method, which has strong anti-interference ability. The method then chooses the appropriate curve model to establish the tram equation, which can extract the tramway accurately and quickly.

2.1 Tramway ROI Segmentation Based on Multilevel Thresholding

The tramway in Suzhou are typically constructed with girder rails. Tram vehicle wheels ride on the rail surface and are held in place by a concave rail with the larger diameter, as shown in Fig. 3(a). The concave rail is called flangeway. In girder rails, the flangeway is part of the cast steel rail, as shown in Fig. 3(c) and (d), which is laid in the street concrete. The characteristics of the tramway we observe the images (e.g. Fig. 3(b)) are as follows:

• (i) The inside of tramway is darker, the outside is lighter, and the difference of gray level is obvious;

• (ii) The darker region is adjacent to the lighter region, and the boundary is clear and smooth.

Figure 3(e) is a flangeway segment that are sampled from the starting positions of left rail on the bottom of Fig. 3(b). Figure 3(f) is its corresponding gray histogram of Fig. 3(e). In these two figures, we can easily observe that the high-end in the range of grey values about [150–200] and low-end in about [0–50] have the obvious peaks.

Fig. 4.

Multilevel thresholding-based tramway ROI segmentation

According to the above characteristics, it is feasible to make use of the darker and the lighter gray values to segment the original image respectively and further extracts the ROIs of potential tramway using distance metric. The specific steps are as follows:

• (i) Image preprocessing. The step includes gray scale processing and smoothing filtering. We define the original image after preprocessing as I(x, y);

• (ii) Multilevel thresholding. We determine two sets of threshold values $T_{dark}$ and $T_{light}$ to deal with the darker and lighter regions, respectively. $T_{dark}$ consists of the minimum threshold value $T_{dark}^L$ and the maximum threshold value $T_{dark}^H$ of the darker region, which can divide pixels into three groups:

  $$\begin{aligned} \begin{array}{l} G_D^{(0)} = \{ (x,y) \in I\;|\;0 \le f(x,y) \le T_{dark}^L - 1\} ;\\ G_D^{(1)} = \{ (x,y) \in I\;|\;T_{dark}^L \le f(x,y) \le T_{dark}^H - 1\} ;\\ G_D^{(2)} = \{ (x,y) \in I\;|\;T_{dark}^H \le f(x,y) \le 255\} . \end{array} \end{aligned}$$

  (1)

  Similarly, we also have three groups of the lighter region using $T_{light}^L$ and $T_{light}^H$:

  $$\begin{aligned} \begin{array}{l} G_L^{(0)} = \{ (x,y) \in I\;|\;0 \le f(x,y) \le T_{light}^L - 1\} ;\\ G_L^{(1)} = \{ (x,y) \in I\;|\;T_{light}^L \le f(x,y) \le T_{light}^H - 1\} ;\\ G_L^{(2)} = \{ (x,y) \in I\;|\;T_{light}^H \le f(x,y) \le 255\} . \end{array} \end{aligned}$$

  (2)

  where f(x, y) is the gray level of the point (x, y).

• (iii) Binarization using multilevel thresholding. In the step, we obtain two binary images using $T_{dark}$ and $T_{light}$ for the darker region and the lighter region respectively, that is, $I_{dark}(x,y)$ and $I_{light}(x,y)$. They are constructed by Eqs. 3 and 4:

  $$\begin{aligned} I_{dark}(x,y) = \left\{ {\begin{array}{*{20}{l}} {0,}\\ {255,} \end{array}} \right. \begin{array}{*{20}{l}} {(x,y) \in G_D^{(0)} \bigcup G_D^{(2)}}\\ {(x,y) \in G_D^{(1)}} \end{array} \end{aligned}$$

  (3)
  $$\begin{aligned} I_{light}(x,y) = \left\{ {\begin{array}{*{20}{l}} {0,}\\ {255,} \end{array}} \right. \begin{array}{*{20}{l}} {(x,y) \in G_L^{(0)} \bigcup G_L^{(2)}}\\ {(x,y) \in G_L^{(1)}} \end{array} \end{aligned}$$

  (4)

• (iv) Morphological dilation. The step is to connect the fracture ROIs of tramway and expand the width of potential tramway. $I_{dark}^{(D)}(x,y)$ and $I_{light}^{(D)}(x,y)$ are the morphological dilation of $I_{dark}(x,y)$ and $I_{light}(x,y)$.

• (v) Tramway ROI generation. We generate the ROI of potential tramway through computing the intersection $I_{ROI}(x,y)$ of these two binary images after morphological dilation, $I_{dark}^{(D)}(x,y)$ and $I_{light}^{(D)}(x,y)$, as shown in Eq. 5.

  $$\begin{aligned} I_{ROI}(x,y) = I_{dark}^{(D)}(x,y) \cap I_{light}^{(D)}(x,y) \end{aligned}$$

  (5)

A complete work flow about multilevel thresholding-based tramway ROI segmentation is described in Fig. 4.

2.2 Adaptive Threshold Parameters Estimation Using Local Accumulation Histogram

The difficulty of multilevel thresholding is to estimate the threshold parameters, that is, $T_{light}^L$, $T_{light}^H$, $T_{dark}^L$ and $T_{dark}^H$. As is known to all, the tram operating environment is complex, traffic scenes are various and the intensity and angle of light always change. Hence, the range of gray levels is not the same, and it is hard to pre-assign a set of experienced thresholds to meet all the scenarios. In this section, we present an adaptive mechanism to calculate these thresholds using the statistic characteristic of local region. The specific steps are as follows:

• (i) The starting points of tramway positioning. When the on-board camera is fixed in front of the locomotive of the tram, its viewing hole is also relatively fixed. The bottom pixels of the image acquisited by the camera are the nearest points to the tram. If we neglect the effect of dead zone (In fact, the length of dead zone (e.g. 30–50 m) can be neglected due to the fast speed of the tram (e.g. 40 mph)), the starting points of tramway always appear on several fixed pixels on the bottomline of the image. Hence, we can determine manually the starting points of tramway.

• (ii) Normalized histogram processing of local image. Given the starting points of tramway, we can easily segment a tramway slice on the bottom of the image according to flangeway edges. Figure 5 gives an ideal illustration of tramway slice on pixel level. The height of local image is h, the widths of flangeway outside and inside are m and n respectively. $f_g(x,y)$ is the local gray image with intensity levels in the range $\left[ 0, L-1\right]$. The normalized histogram of $f_g(x,y)$ is a discrete function of intensity level $r_k$

  $$\begin{aligned} p({r_k}) = \frac{{{n_k}}}{{\sum \limits _{l = 0}^{L - 1} {{n_l}} }}, \end{aligned}$$

  (6)

  where $r_k$ is the kth intensity and $n_k$ is the number of pixels of $r_k$, for $k=0,1,2,\cdots ,L-1$. $p({r_k})$ is an estimate of the probability of occurrence of intensity level $r_k$ in the local image. Assume that the intensity levels of flangeway inside and outside are $r_{fi}$ and $r_{fo}$, we have the probabilities of occurrence of $r_{fi}$ and $r_{fo}$, that is,

  $$\begin{aligned} p({r_{fi}}) = \frac{n}{n+m}, p({r_{fo}}) = \frac{m}{n+m}. \end{aligned}$$

  (7)

• (iii) Threshold parameters estimation. Threshold parameters can be estimated using the normalized histogram of local image. Figure 6 gives an illustration of normalized histogram. A typical normalized histogram of local image has two peaks on the intensity levels of flangeway inside and outside, that is $r_{fi}$ and $r_{fo}$. Ideally, the probabilities at intensity levels of flangeway inside and outside are exactly $p({r_{fi}})$ and $p({r_{fo}})$. Actually, the value of intensity may fluctuate slightly due to the change of illumination and other environmental factors. We hence accumulate the probabilities of the neighboring intensities of $r_{fi}$ and $r_{fo}$ level by level till that Eqs. 8 and 9 are satisfied.

  $$\begin{aligned} \begin{aligned} p({r_{fi}}) +\sum _{i=1}^{i_T}(p({r_{fi}-i})+p({r_{fi}+i})) =\frac{n}{n+m}, \end{aligned} \end{aligned}$$

  (8)
  $$\begin{aligned} \begin{aligned} p({r_{fo}}) +\sum _{j=1}^{\j _T}(p({r_{fo}-j})+p({r_{fo}+j})) =\frac{m}{n+m}, \end{aligned} \end{aligned}$$

  (9)

  At the time, we obtain that

  $$\begin{aligned} T_{dark}^L = r_{fi}-i_T, \quad T_{dark}^H = r_{fi}+i_T, \end{aligned}$$

  (10)
  $$\begin{aligned} T_{light}^L = r_{fo}-j_T, \quad T_{light}^H = r_{fo}+j_T. \end{aligned}$$

  (11)

Fig. 5.

Illustration of tramway slice on pixel level

Compared with the traditional thresholding or edge detection methods [4, 6, 10, 12, 17] background interference and environmental noise can be greatly reduced when normalized histogram is locally accumulated in our approach. Particularly speaking, we inter multilevel thresholding based on the statistic characteristic of local accumulated histogram after normalization. It is the reason why the method can meet real-world requirements and deal with various challenging scenarios.

Fig. 6.

Thresholds estimation on local normalized histogram.

2.3 Tramway Detection Using Region Growing

Given a binary image with potential tramway ROIs, the problem of tramway detection turns to link iteratively different potential tramway segments from some certain starting points. In each iteration, only the segments best to match the required tramway structure based on a set of given criteria would be kept. The region growing algorithm [1] is one of the commonly used methods for solving such the problem. The basic idea of region growing is to use various morphological gradient operators to extract the most approximate pixels in the neighborhood, and predict the position of a following segment using that of current segment [11].

Fig. 7.

Illustration of region growing for tramway detection

In the section, we present our region growing method for tramway detection. The method starts from the binary image obtained by our tramway ROI segmentation. The basis of region growing is some growing seeds. In practice, the starting points in our local binary image are taken as growing seeds. At the beginning, these seeds grow from their exact pixel locations to adjacent pixels depending on a region detection criterion. Next, the detected adjacent pixels are examined using a tracing criterion. If they are similar with their parent seed, we classify them into the set of seeds. It is an iterative process until there are no changes in two successive stages. Finally, interferential lines constructed by sets of few seeds are removed according to prior knowledge.

Fig. 8.

Experiment validation

The key of the method is to draw up a strong anti-interference detection and tracing criterion. The detection criterion we provide is to use some prior knowledge to search the region with potential seeds. The tracing criterion is to use the location and connectivity of the found points to determine their membership to tramway.

The specific steps of region growing are as follows and Fig. 7 give an illustration in a sample binary image:

• (i) Growing seeds determining. In a binary image of size $N \times M$ with tramway points, we search from the bottom row of the image, that is, the $N-1$ th row. The starting points of tramway on the row is known, denoted by a range of pixel points $\left[ (N-1,i), (N-1,j)\right]$. All the pixel points in the range are taken as the growing seeds. If there is no starting point in the $N-1$ th row, the search moves to the $N-2$ th row, and so on.

• (ii) Search region determining for growing seeds of next generation. The growing seeds of next generation can be obtained from every starting points using our tracing criterion. Taking a pixel $(N-1,k)$ as an example, its next generation can be found in the range of $\left[ (N-2,k-w), (N-2,k+w)\right]$ of the $N-2th$ row, where w is the parameter of pixel continuation, as shown in Fig. 7(a). Further, if growing seeds are in a transverse connected region $\left[ (N-1,i), (N-1,j)\right]$ of the $N-1th$ row, the next generation seeds in the $N-2th$ row can be searched in the range of $\left[ (N-2,i-w), (N-2,j+w)\right]$, as shown in Fig. 7(b).

• (iii) New generation of growing seeds. Once the pixel points in the range $\left[ (N-2,i-w), (N-2,j+w)\right]$ are the edge points, it can be considered as new growing seeds, and is also added to the set of trajectory points. Figure 7(c) shows the evolution of growing seeds. If there is no point in the $N-2th$ row, the search along current trajectory line is waived. We repeat this step till all rows are traversed.

• (iv) Interferential lines removing. After searching, several trajectory lines may be constructed. We need to remove interferential lines and pick up the optimal one as tramway. We consider the longest and most complete one of all trajectory lines as the tramway.

• (iv) Tramway equation establishing. Considering the complexity of algorithm and real-time demand, least square piecewise polynomial fitting method is used to establish the equation of tramway. The extracted feature points are segmented in a sequence. The least square method is used to do quadratic fitting with $\mathcal {R}$ points as a piece. When the starting point or the ending point fails to meet the limits, the starting $\mathcal {R}$ points and the ending $\mathcal {R}$ points, respectively, use least square method to do quadratic fitting and extend until the limits. The specific values of $\mathcal {R}$ are as appropriate.

Fig. 9.

Experimental results in different challenging scenarios

Our proposed region growing method can accurately predict the trend of tramway, neglecting the influence of traffic environment, such as the bent, the uphill and the downhill. It exhibits strong anti-interference, uses few calculation and meets the real-time demand.

3 Experiment Validation and Results

Our experiment platform was installed at Tram Line $\#1$ in High-Tech District, Suzhou City, Jiangsu Province of China. The length of the tram line is 18.19 km, and the maximum operating speed of tram is 70 km/h. Our hardware used the camera Hikvision DS-2CD6233F and Intel Core i7 2.5 GHz Industrial computer with 4 GB memory. The frame size of the camera is $1080\times 1920$ pixels and its frame rate is 10 fps. The scope of image collected by the camera is set in the range of [25 m, 250 m]. Our software system was developed in Visual Studio 2010 platform with EmguCV 2.9.0 vision library. The video datasets of all scenarios were collected in the field when the trams were actually running on the ground.

The original image, shown in Fig. 3(b), is taken as an example for experiment validation. Its gray image can be obtained by preprocessing, shown in Fig. 8(a). The preprocessing step consists of image gray-scale and Gaussian filtering. Figure 3(f) is the local image of tramway slice at the starting position of left rail. Its corresponding normalized histogram is obtained to execute our adaptive threshold parameter estimation. We first observe two peaks on the intensity levels of flangeway inside and outside from the normalized histogram, having $r_{fi}=15$ and $r_{fo}=165$. The adaptive method does automatically search and accumulate the neighbors of these two peaks level by level till the sum of histogram bars are equal to the given $p({r_{fi}})$ and $p({r_{fo}})$. The values of $p({r_{fi}})$ and $p({r_{fo}})$ can be induced using prior knowledge of flangeway structure. For Suzhou Tram Line $\#1$, the flangeway outside has a typical width at the top of 37.5 mm plus 12.5 mm, and the flangeway inside has a width of 34 mm. We hence set $p({r_{fi}})=0.405$ and $p({r_{fo}})=0.595$. Using the adaptive algorithm, we get the thresholds: $T_{dark}^L = 9$, $T_{dark}^H = 21$, $T_{light}^L = 139$ and $T_{light}^H = 181$. Figure 8(b) shows the normalized histogram of local image of tramway slice, and the binarization results using $T_{dark}$ and $T_{light}$.

We then conduct the binarization process using multilevel thresholding. Figures 8(c) and (d) are the binary images using $T_{dark}$ and $T_{light}$ for flangeway inside and outside respectively. In order to connect the fracture ROIs of tramway and expand the width of potential tramway, a morphological dilation operator is used. Finally, we generate the image with the ROIs of potential tramway through computing the intersection of these two dilated binary images, as shown in Fig. 8(e). Given a binary image with potential tramway ROIs, our region growing method is done for detecting the tramways. The initial growing seeds are picked up from the binary image. The parameter of pixel continuation is set to 7, that is $\omega =7$. The red line in Fig. 8(f) is the induced tramway model through region growing and least square piecewise polynomial fitting.

Table 1. The comparison of computational time that tramway detection spends in different scenes (ms)

To verify the effectiveness of the proposed method, we perform more experiments in which we apply our algorithm to different challenging scenarios. Figure 9 shows the experimental results in different scenes, including original image, binary image after multilevel thresholding segmentation and the region of interest between railways. The scenarios of Fig. 9(a), (b) and (c) are the bent of natural grassland with ordinary illumination, the downhill of artificial grassland in rain and fog, the uphill of asphalt road in the evening. The experimental results show that the proposed method can not only accurately detect the straight tramway, but also work for the curve, uphill and downhill. Further, it not only can be applied to the general scenario in ordinary illumination, but also has good sensitivity and accuracy for varying weather, for example, at night, in rain and fog and on various ground types. We also investigate the performance on time consuming for practical application. Table 1 gives the comparison of computational time that our method spends in different scenarios. According to the table, the scenarios in the night, in rain and fog, and under other special scenes need more time to detect tramway. Thus it can be seen that accurate recognition in challenging scenarios is at the expense of a slight increase in time, which is in a controllable range. In the real-world system, we use the technique of dynamic thread pool, which can flexibly increase or lessen threads to satisfy real-time.

These results prove that the method can effectively detect tramway, accurately extract the ROI of tramway. The method shows good robustness, which can deal with different scenes and meets the demand of practical application.

4 Conclusion

Tramway detection is important but difficult due to various challenging situations in real-world urban rail traffic scenarios. In this paper, we propose a video-based tramway detection approach to address the issue. Our approach can efficiently segment the ROIs of tramway using a multilevel thresholding method with adaptive parameters estimation using local accumulated histogram. Instead of recognizing pixel by pixel on every video frame, which is believe to be time-consuming and infeasible for autonomous driving, our proposed method uses an improved region growing scheme to extract the optimal tramway. In addition, we use only a simple setup with a surveillance camera plus adaptive algorithm without rely on expensive equipment. Our method has been successfully tested in realistic urban environment, which proves that the accuracy and real-time performance of tram track detection in different challenging scenarios.