Keywords

1 Introduction

In area of steel manufacturing the control of processing hot metal sheets, strips and rods is not a trivial task due to nonlinearities and complexities of hot metal rolling within extremely hostile environment that frequently prevents the appropriate location of sensors to measure process features in real-time [1, 2]. Particularly, a forced braking process of rolled rods on the cooler is important and challenging task of automatic control in modern continuous small-section mills. The efficient solutions of management of steel production provide reducing the amount of scrap and increase a quality of rods with small cross-section profiles of diameter from 10 to 16 mm [3, 4].

Recently, the considerable efforts of researchers are addressed to develop high speed, accurate and low cost control equipment for metal processing on mills by using mechanical, electrical, hydraulic tool, smart optical, laser or X-ray sensors [5,6,7,8] However, the problems of accuracy of automating mechanisms used for stabilization of valve diverter of hot metal rods on cooler and control of their forced braking remain unsolved and require further research. The main technological factors that introduce disturbance to braking rods during their continuous rolling on mill are fluctuations of transfer velocity and variable feedback time delay of control tool for dynamic deceleration. Moreover, the velocity of metallic straight rods is affected by changes of their geometric dimensions, temperature, tension and strain [2, 6, 9]. So, the most difficult problem of automatic control is to provide such braking, when rod front ends after complete stoppage on cooler would be aligned in one plane (Fig. 1a).

Fig. 1.
figure 1

(a) Arrangement of rods on the cooler (fragment, top view); (b) Rods on cooler of light-section mill 250-1 PJSC ArcelorMittal.

Full size image

The principal goal of aligning the front ends of rods after their stoppage on cooler is significant reducing metal waste trimmings after cutting-to-length packages of rods on stationary shears. The aligning front ends of the rods in the package cannot be provided using fixed detent due to low bending stiffness of small rolled profiles.

In well-known mill machinery management systems the effective control action on the value of braking distance without interfering with the process of separation of adjacent rods on valve diverter is a forced deceleration generated by brake electromagnets placed in the receiving tray in the straightening trench of cooler [2, 7,8,9]. That also allows less severe thermal regime of the electric coils of brake electromagnets and does not require electric power supply of electromagnets to be mounted on the moving parts of mil [1, 3, 6]. However, disadvantages of recent implementations of mentioned control are the difficulty of sensor location near high temperature mill (up to 400°C), sensor unstable functionality and distortions of acquired from them information in environments with strong vibration and shock. The proposed solution of this problem exploits visual remote diagnostic, monitoring and control of rod braking process on hot mill based on specific methods of digital image processing and pattern recognition.

This paper is organized as it follows. In the Sect. 2 an overview of recent trends and new advances in computer vision is presented and some appropriate image processing approaches are selected. In the Sect. 3 the proposed algorithm for monitoring rod deceleration on hot rolling mill is introduced. Finally, the discussion of results obtained in tests and evaluation of the proposed approach are presented.

2 Image Processing Approaches for Industrial Applications

The main idea of alternative approach for controlling parameters of rod braking process on hot rolling mill consists in using methods of video processing acquired from remote vision systems. Recently, industrial use of optical measuring systems is common practice for monitoring production lines for steel manufacturing on rolling mills and cut-to-length lines [7, 8]. For solution of the particular task of managing technological operations on light-section mill 250-1 PJSC ArcelorMittal [10], the following technical requirements have to be taken into account. Hot metal rods arrive to cooler receiving tray in pairs with the speed about 0.1–0.2 m/s. The distance to provide forced braking rolled rods on cooler with deceleration 2.5–3.5 m/s2 is less than 2 m. The accuracy of aligning front end of stopped rods must be less than 0.1 m. Stopped and aligned rods on the cooler are moved in the transverse direction for later formation of packs of 30–40 rods as it is shown in Fig. 1b).

The solution must solve: (1) real-time tracing thin rods in video by detection of their front ends; (2) generating continuous action signals for magnetic brakes on base of computing frame-by-frame deceleration; and (3) providing aligning rods by their standstill in one plane. In this paper the solution of 1st problem is presented.

There are some well-known techniques for estimation of motion characteristics such as displacement vector, velocity, acceleration, motion oscillation, errors of prediction, etc. This is a known practice to use optical flow or motion field for describing position of moving objects in images. The principal disadvantage of these approaches is the computational cost that makes difficult their utilization in real time applications [11, 12]. Another technique for quantitative estimation of motion characteristics is based on the block correspondence, when the same pattern is used within consecutive frames as reference. It allows overcoming a problem of progressive increment of compared patterns but aggregates accumulative error proportional to time function [13].

The most widely used approaches for pattern recognition may be subdivided in knowledge based, feature invariant, template matching and appearance-based methods. The methods of the 1st group may be used for detection and tracking of rods in motions because they consider a region of interest as whole entity and try to localize its core components. However, they provide high accuracy and speed only under controlled conditions in structured environment with well-established formalism for knowledge representation and reasoning [14].

The 2nd group of techniques uses local features of single pixel or small region (color, gradient, gray value, dimensionality, texture, etc.) invariant to variation of illumination, noise, scale, relative position, orientation and changes in viewing direction providing robust pattern recognition under different conditions. They are known as scale invariant feature transform (SIFT), difference of Gaussian points (DoG), entropy based salient region detector (EBSR), corner detectors, intensity and edge based regions, mathematical morphology and gradient based approaches. In is evident that for detection and tracking rods in motion this group of method is well fitted particularly, when recognition of patterns is not necessary as well [15, 16].

From other hand the template matching techniques involve evaluation of correlation of predefined template to test image. Unfortunately, for these approaches some factors like tolerance to deformation, robustness against noise and feasibility of template matching during image distortion must be taken into account, because they are sensitive to occlusions and variations within template [17, 18].

Finally, there are several prospect local or global appearance-based object recognition methods. The local approaches are used to process regions of interest characterized by corners, edges, shapes while the global ones use color, entropy, gradient moments and sometimes semantics. Particularly, the global approaches transform whole input image onto suitable lower dimensional data set that does not require complex time consuming processing. Unfortunately, the encoding size of dimensional data sets are enlarged considerably as well as used iterative training and evaluation processes limit their use in real-time applications [14, 19].

3 Proposed Algorithm for Detection and Tracking Rods on Mill

Due to specific technical requirements, the principal objective of this paper is to develop high-speed image processing algorithm for detection and feature extraction of rods in motion in images with variable illumination and presence of noise obtained from standard video camera with resolution no more than 1920 × 1080. The following proposed algorithm provides detection front-end of rods during their deceleration.

An original video must be adjusted to eliminate camera displacement if there is not fixed position of image capture. For this task a video stabilization algorithm was used, eliminating the black boxes generated by the stabilization algorithm reducing frame resolution to 600 × 400 pixels (Fig. 2a). The image frames \( I_{600 \times 400} \) are converted to gray scale applying then sharpening filter producing image \( I_{sharpen} \) shown in Fig. 2b).

Fig. 2.
figure 2

(a) Original image \( I_{600 \times 400} \) with reduced resolution of 600 × 400 pixels; (b) Gray scale image \( I_{sharpen} \) after applying sharpening filter.

Full size image

After multiple experiments and due to very good image contrast of rods in motion over cooler tray (well separated objects in bimodal histogram of the scene), the binarization with empirically defined threshold is applied for generation a binary image \( I_{bin} \) shown in Fig. 3a). Some morphological filters are applied with the proposal to reduce the noise of image and separate connected rods. For that task the following structural elements SE (see Eqs. 1 and 2) were used for: (1) Opening with vertical structuring element (SE_ver); (2) TopHat with vertical structuring element (SE_hor); (3) Thickening with diagonal structuring element of type 1 (SE_diag1); (4) Thickening with diagonal structuring element of type 2 (SE_diag2).

Fig. 3.
figure 3

(a) Binary image \( I_{bin} \) obtained by applying empiricaly defined threshold; (b) \( I_{morpho} \) is a result of applying morphological operations to binary image.

Full size image
$$ SE\_hor = \left\{ \begin{aligned} \begin{array}{*{20}c} 0 & 0 & 0 \\ \end{array} \hfill \\ \begin{array}{*{20}c} 1 & 1 & 1 \\ \end{array} \hfill \\ \begin{array}{*{20}c} 0 & 0 & 0 \\ \end{array} \hfill \\ \end{aligned} \right\};\begin{array}{*{20}c} {} & {SE\_ver = \left\{ \begin{aligned} \begin{array}{*{20}c} 0 & 1 & 0 \\ \end{array} \hfill \\ \begin{array}{*{20}c} 0 & 1 & 0 \\ \end{array} \hfill \\ \begin{array}{*{20}c} 0 & 1 & 0 \\ \end{array} \hfill \\ \end{aligned} \right\}} \\ \end{array} ; $$
(1)
$$ SE\_diag1 = \left\{ \begin{aligned} \begin{array}{*{20}c} 0 & 1 & { - 1} \\ \end{array} \hfill \\ \begin{array}{*{20}c} 0 & 0 & 0 \\ \end{array} \hfill \\ \begin{array}{*{20}c} { - 1} & 1 & 0 \\ \end{array} \hfill \\ \end{aligned} \right\};\begin{array}{*{20}c} {} & {SE\_diag2 = \left\{ \begin{aligned} \begin{array}{*{20}c} { - 1} & 1 & 0 \\ \end{array} \hfill \\ \begin{array}{*{20}c} 0 & 0 & 0 \\ \end{array} \hfill \\ \begin{array}{*{20}c} 0 & 1 & { - 1} \\ \end{array} \hfill \\ \end{aligned} \right\}} \\ \end{array} ; $$
(2)

The results of morphological operations are shown in image \( I_{morpho} \) in Fig. 3(b), where rods on cooler are not yet correctly separated. So, an algorithm for analysis of discontinuities between them is proposed. For better visual inspection the rods are considered as particular regions painted in different colors using fast blog generation method based applying L-shape connected components. In the Fig. 4(a) founded connected components are shown as sets of regions of different colors. For better rod separation each detected vertical red line, which corresponds to particular rod, is evaluated looking for rod “adhesion” with adjacent ones. The pixels that represent adhesions (connection) between separated lines are detected and later are eliminated.

Fig. 4.
figure 4

(a) Some color regions with connected components; (b) \( I_{discon} \) with detected positions of horizontal sections with pixels indication connections between rods. (Color figure online)

Full size image

During evaluation of pixels along each line that corresponds to particular rod, the sections with continuous horizontal sets of pixels of the same color are found indicating the presence of adhesion between adjacent rods as it is shown in Fig. 4(a). These horizontal sets of pixels may be considered as discontinuity of the red lines, which represent separated rods. In Fig. 4(b) the red lines indicate the start of search for each rod; the pink lines indicate the path while the absence of horizontal sections is detected and the yellow segments show the position, where there is adhesion (connection) between two rods. Yellow rectangles from Fig. 4(b) are used as masks representing the regions where rods have discontinuity. The positions found as discontinuities in Fig. 4(b) is shown as masks in Fig. 5(a). The difference \( I_{morpho} \) between the image obtained after application of morphological filters \( I_{morpho} \) and discontinuity mask \( I_{separation\_mask} \) is computed and the regions forming individual rods are successfully separated as shown in Fig. 5(b). In Fig. 6 the image with color regions corresponding to detected and separated rods are shown.

Fig. 5.
figure 5

(a) Mask \( I_{separation\_mask} \) with detected discontinuities for connected rods; (b) Binary image \( I_{difference} \) of successfully separated individual rods.

Full size image
Fig. 6.
figure 6

Image with color regions corresponding to detected and separated rods. (Color figure online)

Full size image

The same procedure for detecting rod front ends computing their discontinuity is used. In contrast to detection of horizontal sections of rod adhesion, for estimation of their front ends \( I_{frontend\_difference} \) discontinuity is searched with respect to black background as shown in Fig. 6.

Obtained from images information is useful for generating control signals in rod deceleration braking system because at each moment the exact location of front ends of rods in motion is known. So, deceleration of rods now may be provided by iterative computing of their front ends until the final stop aligning them in one plane.

4 Discussion of Conducted Tests and Evaluation of Obtained Results

Several tests of the proposed algorithm and the designed visual system for monitoring of hot metal rods have been carried out on the continuous light-section mill 250-1 PJSC ArcelorMittal. For image sequence acquisition a conventional Cannon video camera with resolution 1920 × 1080 and 30 frames per second has been used. The location of camera with respect to receiving tray of cooler is shown in Fig. 7.

Fig. 7.
figure 7

Video camera location with respect to cooler in conducted tests.

Full size image

Only two rods simultaneously arrive to cooler and must be stopped and removed from diverter valves of roller conveyor but in conducted tests all rods are processed.

A complete execution of the proposed algorithm is shown in Fig. 8 where the processing times for each step and finally for whole procedure are shown for different resolution of input images particularly, for 320 × 240 (with complete execution time equal to 0.012 s), 600 × 400 (0.055 s), 720 × 576 (0.61 s) and 1920 × 1080 (0.91 s).

Fig. 8.
figure 8

Examples with algorithm execution and processing times for each step and for whole process for different resolution of input images (320 × 240, 600 × 400, 720 × 576 and 1920 × 1080).

Full size image

The average time of detecting rod front end by the proposed algorithm with selected resolution of 600 × 400 is less than 0.055 s achieved on the computer with Core Duo processor of 3.3 GHz and 4 GB of RAM. Therefore, during one second the algorithm may process 18 frames. If the initial rod velocity on the cooler on the last meter of deceleration is about 0.1 m/s, the rod requires at least 5 s for complete stop. During these 5 s the algorithm processes 18 × 5 = 90 frames providing analysis of each 1.1 cm of rod displacement. The required maximum error of front end alignment after complete stop of rod is equal 10 cm; however, the capability of the proposed algorithm to operate with error about 1 cm is completely satisfactory. For processing images with resolution of 720 × 576 the algorithm provides correct analysis of rods in positions separated by 6.5 cm, even processing images with resolution of 1920 × 1080 gives acceptable alignment error of 10 cm.

For evaluation of precision of rod detection the matching strategy using Euclidean distance between coordinates of rod front end points found by algorithm and located in input images has been applied to more than 300 processed images for each of multiple recordings. The quantity of similarity of front end point coordinates between two images is defined by Eq. (3):

$$ similarity(rod\_frontend\_position) = \sum {match(} I_{input} ,I_{frontend\_difference)} $$
(3)

The average precision of detection of rod front ends by algorithm for daylight and artificial illuminations and different resolution of input images is resumed in Table 1.

Table 1. Average precision of rod front end detection for different resolution of input images.
Full size table

It has been noted that the higher resolution of images does not provide significant improvement in the performance of algorithm but can interfere with timing parameters required for generation of the controlling signals for rod braking system. Despite of simplicity of the proposed algorithm, it provides quite acceptable low complexity processing however, the speed of rod detection and tracking during deceleration may be improved by eliminating double (horizontal and vertical) computing discontinuities and processing only small region of interest in image, where rods in motions are appeared. The improvement of the processing approach may also be done using second camera with viewing direction perpendicular to rod translation on a mill.

5 Conclusions

In this paper a specific computer vision application for inspection of steel manufacturing on hot rolling mills is presented. The proposed algorithm provides visual remote monitoring rod braking process on mill based on real-time detection and tracking thin rods in sequences of images by analysis of their front ends during deceleration. It exploits morphological filtering and discontinuity masks for finding and separation of rods on rolling mill as well as provides fast enough detection and tracking of rod front ends in cooler with precision in range of 90–96% on artificial and 92–98% on daylight illumination, respectively processing images with resolution from 320 × 240 to 1920 × 1080. The information obtained from analysis of rod deceleration during frame-by-frame image processing will be used to provide solutions for control of some processes particularly, for generation of continuous action signals for magnetic brakes as well as it will be used to align rods by their standstill in one plane reducing metal waste trimmings after cutting-to-length packages of rods.