Wave Front Tracking in High Speed Videos Using a Dynamic Template Matching

Abstract

In recent years, image processing has been evolving as an important tool to estimate the properties of a generated wave during a gas experiments. The wave properties were estimated by tracking a wave fronts in the images of a high speed video, captured during the experiment. In this work, we purposed a dynamic template matching method, based on the mean square error (MSE) between the intensity values of a template and its foot print in an image to track the wave front. At first, a dynamic template of a predefined size [5 $\times$ 20] was created, whose values varies according to the minimum and maximum intensity of the considered image. Secondly to reduce the processing time, a bounding box was set around the area of interest in the considered image, such that the matching process is limited within the area covered by the bounding box. The purposed method was tested for four different high speed videos from four different gas experiments conducted two different experimental set-ups. All the results from the purposed method are within an acceptable accuracy.

1 Introduction

Template matching or pattern matching is an image processing method to detect the desired object in an image by using a predefined template. A template matching starts with creating a template of relatively smaller size whose one or multiple features matches with the features of the desired object. Then, the created template is slid in a pixel-by-pixel basis, computing the similarity between the template features and its footprint in the image [1]. The few common features that are used for calculating a similarity while matching are, normalized cross correlation (NCC), the sum of absolute difference (SAD), the sum of squared error (SSD), mean square error (MSE) [2]. In this work, we took the template matching based on MSE, due to its simplicity and fast processing. The MSE takes a mean of the squared difference between the intensity of each pixel in the template and the corresponding pixel in its footprint in the image. Assuming template T of size [m $\times$ n] slides over an image I, then at each position (x,y) in I, MSE is estimated as in (1). The template matches the best in the image pixel where MSE is the minimum.

$$\begin{aligned} MSE(x,y)= \dfrac{1}{no. of pixel}\sum _{k=1}^{m} \sum _{e=1}^{n} [I(x+k,y+e)-T(k,e)]^{2}. \end{aligned}$$

(1)

In general, a gas explosion can be defined as a process where combustion of a premixed gas cloud, i.e. fuel-oxidiser is causing a rapid increase in pressure in a close vessel or confined area due to any kind of external energy. When the pressure exerted inside the vessel is higher than the vessel can hold then it explodes producing extremely powerful and destructive waves [3]. The most common approach to study these geberated waves and estimate their characteristics are by using the pressure transducers in the experimental area and/or computer simulations [4]. However, in recent years, the estimation of wave characteristics using the high speed videos and image processing have been emerging [5,6,7]. A high speed video captured during the gas experiment was later processed using image processing technologies to track the wave front in the images of the video. By tracking the wave front in these images, the wave characteristics like speed, temperature and pressure were estimate.

Fig. 1.

Block diagram of the framework.

However, due the poor quality of images in a high speed videos, most of the image processing framework consists of multiple processing units, which not only cost a computational time but also require manual interference. One of the previous work that used template matching for front tracking can be read [8]. The tracking was done in the raw images with two predefined templates: one for a straight wave and one for the tilted (oblique wave) in a single image (refer to Fig. 2 second row). The framework then use a post-processing for finding the optimum result from both templates. The matching was based on the predefined average intensity of the template for an individual video. Even though, the framework works fine with some manual input, the sliding of a template in an overall image along with post-processing takes a high computational time. In this work, we purpose a new method of creating a dynamic template and sliding the created template within a certain region in an image bounded by a bounding box. As, we are working with a set of images from a multiple high speed videos, whose intensities varies with each image, we create a template with a fixed size but with a varying value depending in the intensity of each image to be matched.

Fig. 2.

Some of the images from different gas experiments chronologically sorted from left to right. An arrowhead points towards the direction of the wave propagation in the corresponding video. From top to bottom row: $N_{2}$, $CO_{2}$, $H_{2}$ and $H_{2}+ air$ experiment.

The rest of the paper is organized as follows. Section 2 gives a brief description on the high speed videos and the methodology of template creation, setting bounding box and matching the created template in the area surrounded by the bounding box. The results from the purposed method with some discussion are presented in Sect. 3. Lastly, the conclusion and some possible further work is discussed in Sect. 4.

2 Materials and Methods

The block diagram presented in Fig. 1 depicted the work flow of the overall framework. More on the each part are described in following sub-sections.

2.1 High Speed Videos

The four high speed videos captured during four different gas experiments were processed in this work. The experiments were conducted with $N_{2}$ (Exp.2534), $CO_{2}$ (Exp.2558), $H_{2}$ (Exp.0016) and $H_{2}+ air$ (Exp.0022) gases. Figure 2 shows the wave propagation during the considered four gas experiments. The images in each row are from an individual experiment and are presented in chronologically order from left to right. All the upcoming figures in this paper with four rows follows the structure row wise i.e. from top to bottom row: $N_{2}$, $CO_{2}$, $H_{2}$ and $H_{2}+ air$ experiment. An arrowhead points towards the direction of the wave propagation in the corresponding video. The characteristics of the generated wave during any gas explosion depends on the characteristics of the gas itself and the initial conditions like initial pressure and temperature. Due to this reason, the structure of wave is different in each experiments. The objective of this work is to design a single framework, which track the wave front in all these images/videos regardless of their structure.

Fig. 3.

Background image created for left: $CO_{2}$ and right: $H_{2}$ experiments.

The first two experiments were conducted in a shock tube and the propagating wave is known as a reflected shock wave. The details of a shock tube and the method of capturing high speed video while experimenting in a shock tube can be read in [5]. The bottom two experiments were conducted in an open end experimental tube and the propagating wave is known as a detonation wave [10]. A schlieren technique [12] of imaging was used for capturing all the videos, and a special high speed camera called ‘Kirana’ was operated with the frequency of 500,000 frames per second. Each high speed video consists of 180 images of size [768 $\times$ 924] pixels $\approx$ [70 $\times$ 100] mm. However, some part of the image contains capturing window, which was cropped appropriately before processing. The final size of images for respective experiments are: $N_{2}$ and $CO_{2}$ is [400 $\times$ 910], $H_{2}$ is [356 $\times$ 631] and $H_{2}+ air$ is [356 $\times$ 640].

2.2 Pre-processing/Filtering

Due to the high frame rate of the camera and the ongoing chemical reactions during the experiment, the images in the high speed videos are generally with high amount of noise. Therefore, it is preferred to pre-process the images before further processing. The pre-processing of an image in the current framework consists of a background subtraction followed by a low pass frequency filtering. For background subtraction, a background image was created by taking average intensity values of the initial images which doesn’t consists of the visual wave front. The background image created for the high speed videos of $CO_{2}$ and $H_{2}$ experiments are shown in Fig. 3. After that, the created background image was subtracted from each image with the visual front or which needs to be procressed. The remaining noise in the background subtracted image was then removed by using a low pass filter with the threshold frequency of 50 Hz. The examples of the background subtraction in the images from high speed videos is presented in Fig. 4 left column, and the right column shows their respective low pass filtered images. The process of the frequency filtering in the images and its benefits can be read on [15].

Fig. 4.

Result of pre-processing, right column: result after background subtraction, left column: respective filtered image after LPF.

Fig. 5.

The mesh plot showing the intensities. Left: the filtered images and right: a template created for the corresponding images (right).

2.3 Template Creation

The close look of the intensity difference between wave band and background in the background subtracted images suggested that the waveband contains of bright pixels where as background contains of dark pixels. With this reference, a dynamic template of predefined size [5 $\times$ 20] was created, which values depends on the minimum and maximum intensity of the considered image. One half of the template, contains the minimum intensity value which should technically be the intensity of a background, while another half contains the maximum intensity value which should technically be the intensity of the wave band. However, which side of template takes the minimum and maximum value depends on the direction of the wave propagation. For example, for first two videos in which the wave is propagating from right to left, the wave front is located at the left side of the wave band. Therefore, the right half of the template contains the maximum intensity value (wave) and left half contains minimum (background). The values in the template for the bottom two videos would be exactly opposite as the wave is propagating in an opposite direction and the wave front lies at the right side of the wave band. The mesh plot illustrating the intensity level of a wave band and background in the filtered images from Fig. 4 along with the mesh plot of their respective created templates are presented in Fig. 5.

Fig. 6.

Template matching process left: a filtered image from $CO_{2}$ experiment with a bounding box for matching, center: MSE calculated from matching within the bounding box, right: the result of choosing the minimum MSE in each row presented by the green curve in a raw image of the same image in left. (Color figure online)

2.4 Bounding Box

As it can be seen in all the images presented in previous sections, the wave actually stands in the small part of image. Hence, sliding a template all over the image will only increase the processing time. Therefore, to minimize the processing time, a bounding box was created around the area of interest (wave) such that the sliding of template occurs only inside the bounding box. The height of bounding box is always the number of rows in the image, whereas the width varies depending on the horizontal span of wave band in the image. For the initial image, the bounding box was set at the side of image where, the wave originates and then it moves along the direction of wave propagation with each consecutive images. The movement of the bounding box was governed by the values of the previous tracked front.

Lets take an example of the $CO_{2}$ experiment, the bounding box was set initially at the right end of the image where the wave originated. The right end of the bounding box was set at the right end of image itself, whereas the left side was set 200 columns ahead of the right end. After the first front was tracked, the position of the bounding box was updated according to the position of the tracked front. As the wave was always going forward (towards left), the right side of the bounding box was updated with the median value of the first tracked front while keeping the width of the bounding box constant. The choice of 200 columns as the width of the bounding box came from the a priori visualization of the high speed video which gave rough idea about the total wave span (refer to Fig. 2 second row). Similarly for the $H_{2}$ experiment, the bounding box with the width of 100 was initially set at the left side of the image. After first tracking, the left side of the bounding box was then updated with the median of the first tracked front keeping the width as it is. However, in $H_{2}+ air$ experiment to accommodate the shape of the wave, the bounding box of width 200 was set with the median value of last 50 rows. One of the raw images from $CO_{2}$ and $H_{2}$ experiment are presented in the left image of Figs. 6 and 7 respectively. The width of bounding box for the $N_{2}$ experiment are 50.

Fig. 7.

Template matching process left: a filtered image from $H_{2}$ experiment with a bounding box for matching, center: MSE calculated from matching within the bounding box, right: the result of choosing the minimum MSE in each row presented by the green curve in a raw image of the same image in left. (Color figure online)

2.5 Template Matching

The sliding of the template always started from the top left of the bounding box and moved towards top right. For example, a bounding box in Fig. 6 spreads from top to bottom (all rows) and column 350 to 550. The first matching took place at top left of bounding box i.e. $1^{st}$ row and $350^{th}$ column of the image and the template slide along each column calculating MSE at each pixel till $531^{th}$ column. After, it reached $531^{th}$ column, it slide to $2^{nd}$ row $350^{th}$ column and continued till $531^{th}$ column and so on. Four pixels at the bottom and 19 pixels at left of bounding box was exempted due to boundary adjustment. At each position, MSE between the template and its footprint in the image was estimated as in (1), such that $x = 1:396, y = 350:531$. After completing calculating MSE in one image, a pixel with the least MSE was picked out in each row. Please note that the actual front position is in the middle of the template, such that the minimum MSE pixel in any row gives the front position at 2 rows below. The position of the wave front is then 10 columns behind actual column of MSE pixel. For example, the minimum MSE point in a $1^{st}$ row is at lets say column $y_{m}$ actually gives the front position in $3^{rd}$ row which will be in column $y_{m} + 10$. Hence, there was no front tracked for top 2 and bottom 2 rows.

Fig. 8.

Template matching process left: a filtered image from $H_{2}$ experiment with a bounding box for matching, center: MSE calculated from matching within the bounding box, right: the result of choosing the minimum MSE in each row presented by the green curve in a raw image of the same image in left. (Color figure online)

3 Results and Discussion

A created bounding box of width 200 pixels, the result of template matching within the bounding box and the position of front in one of the raw image from $CO_{2}$ experiment is shown in Fig. 6. Similarly, the image from $H_{2}$ experiment with a bounding box of width 100 pixels, the calculated for the bounding box and result from the minimum is presented in Fig. 7. Figure 8 summarizes the results of a dynamic template matching based on MSE for tracking wave fronts in four different high speed videos. The first and second column presents the result in an individual images while, third columns shows all the tracked fronts with their respective position in the image. High speed video of $N_{2}$ and $CO_{2}$ are comparatively with less background noise than the $H_{2}+ air$ and $H_{2}$ experiment. The tracked fronts are therefore with less or no distortion as seen in top two rows. However, few distortions can be seen in the results from $H_{2}+ air$ experiment, the noise within the bounding box is matched more than the actual front. In such cases, some post processing should be performed, as simple one can be the smoothing of the front or piecewise line fitting.

The framework/matching process also tested in the raw images as well as the background subtracted images from the high speed videos. For better illustration in Fig. 9, one image from each experiment are presented in a raw, background subtracted and filtered form with the respective front tracked by the method in the same images. For better quality video with $N_{2}$, the results are almost same for all type of images. For the $H_{2}$ experiment, the results are smoother and better with each step of pre-processing. In contrast, for $H_{2}+ air$ and $CO_{2}$ the worst results is while using background subtracted image as the noise in the background subtraction enhanced a background noise as line structure inside the wave band.

Fig. 9.

Template matching process left: a filtered image from $H_{2}$ experiment with a bounding box for matching, center: MSE calculated from matching within the bounding box, right: the result of choosing the minimum MSE in each row presented by the green curve in a raw image of the same image in left. (Color figure online)

Figure 10 shows the result of using ‘prewitt’ edge detection method from Matlab image processing toolbox corresponding to right bottom image in Fig. 4. Some of the available edge detection methods in various processing toolboxes were able to detect the edges, however they did not provide the required precision of front position. The red curve is plotted with the first white pixel from right and green one is from the template matching. This shows the importance and the advantage of using a robust method like template matching in order to track the exact front position.

4 Conclusion

A dynamic template with the predefined size and varying value was created to track the wave front the high speed videos. A bounding box which size varies with the size of the wave in the image was set in the images for sliding the created dynamic template to minimize the processing time. The use of bounding box has minimize the processing time by more than 1/3 times. The time of execution of one image from $H_{2}$ is 0.440 s while using bounding box while took 1.839 s while not using bounding box. However, the setting and the movement of the bounding box depends upon the wave propagating direction and structure of the wave, so a prior information about the structure of the wave in an image is necessary. Visually, all the results are within a acceptable accuracy and a purposed method of template matching have a huge potential for tracking various kinds of wave. The purposed method can also be used without any pre-precessing, however, the tracked fronts would be rougher hence, some post processing are suggested.

Fig. 10.

The result of using ‘prewitt’ edge detection method corresponding to right bottom image in Fig. 4. Red curve - first white pixel from right and green curve - the template matching. (Color figure online)