Local spatial correlation-based stripe non-uniformity correction algorithm for single infrared images

https://doi.org/10.1016/j.image.2018.12.005Get rights and content

Highlights

  • Proposing a more accurate and simple single image stripe non-uniformity correction method for Infrared images.

  • Making use of the spatial correlation between pixels in the same column.

  • Adopting an edge detection method to avoid edge blurring.

Abstract

Stripe non-uniformity typically exists in infrared images and affects the visual effect; thus, eliminating stripe non-uniformity is essential to improve image quality. In this paper, a correction model with higher accuracy is developed. Unlike some other stripe non-uniformity correction methods using the same gain coefficient and offset parameters for the pixels of each column, different deviations of the correction parameters in the same column resulted by unsatisfactory preliminary non-uniformity correction are considered and are thought to be small and still relevant in space. The proposed method calculates the correction parameters for each pixel respectively based on the intrinsic spatial correlation between adjacent pixels in a column. What is more, an edge detection method is included. The experimental results indicate that the proposed algorithm effectively eliminates stripe noise of images of different scenes and it also works well in terms of preserving details. Furthermore, the algorithm exhibits high real-time performance.

Introduction

Recently, with the rapid development of electronic technology, infrared imaging technology is significantly used in military and civilian applications [1]. However, the infrared image quality is considerably affected by the non-uniformity of the infrared focal plane array (IRFPA). Ideally, the response of each detector in the IRFPA is equal when each detector receives a uniform radiation. However, in real applications, despite receiving identical incident radiation, the response of each detector varies due to the influence of the manufacturing process and materials, and this further leads to the non-uniformity of infrared images [2], [3].

Several non-uniformity correction (NUC) methods were proposed to date and are roughly divided into two types as follows: scene-based and calibration-based [4]. As indicated in extant studies [5], [6], scene-based NUC methods evaluate the gain and offset of the detector unit based on the changes of scene information in an image without the radiation reference source. However, a few of the methods require the storage of a significant number of images. Additionally, they may not work well when the blurriness caused by motion is considered [7]. Calibration-based NUC methods are more effective and easier to conduct when compared with scene-based algorithms. With respect to the methods, the most common and classic NUC solution corresponds to two-point calibration that obtains the gain coefficients and offset parameters by illuminating the focal plane array (FPA) with two uniform radiation intensities. Nevertheless, it is not possible for the methods to adaptively trace the response drift of detector units with changes in the surrounding environment and FPA temperature. In real applications, a shutter is used as an external uniform temperature source to regularly calibrate and update the correction parameters [8]. However, regular applications of the shutter temporarily interrupt the imaging process, thereby leading to the discontinuous operation of IR cameras. In order to overcome the defects of existing methods, a few shutter-less methods [8], [9], [10], [11] were proposed in recent five years. However, the methods tend to require an extremely tedious calibration procedure and do not achieve satisfactory correction results.

Stripe non-uniformity is a special type of non-uniformity that is extremely typical in uncooled starring IRFPA. Based on CMOS architectures, an uncooled infrared focal plane includes multiple amplifiers [12]. In order to reduce manufacturing costs, the detectors of the same column share a readout channel and a corresponding amplifier, blind bolometer, accumulator and analog-to-digital converter (ADC). Thus, in ideal situation, it is sensible to set the same correction parameters for the same column. The differences between the amplifier factors, reference bolometers and ADCs and noise of the bias voltage in infrared FPA are the fundamental causes of stripe non-uniformity [13].

As stripe non-uniformity is a type of non-uniformity, it is apparently eliminated to a certain extent by traditional NUC although special methods should be developed if higher image quality is required since traditional NUC techniques experience difficulties in terms of completely removing stripe noise. Thus, stripe noise is still recognizable in the images shown in Section 3 (the experiments section). Additionally, the methods are termed as stripe non-uniformity corrections (SNUC) [14], [15] and are used to improve infrared image quality.

As indicated in previous studies, several methods aimed at SNUC were developed [16], [17], [18], [19], [20], [21], [22], and most of them tend to use the characteristics of stripe non-uniformity to correct the non-uniformity of images. With respect to each column, all the pixels in it exhibit the same correction parameters in the methods. For example, a minimum mean square error method for SNUC (MMSE) [17] and a stripe noise removal method based on the gray-scale co-occurrence matrix (GSCM) [18] was proposed by Qian et al. The goal of the MMSE method involves determining the optimal SNUC parameters that make the corrected image closest to the ideal image. However, given that the correction parameters for an entire column of pixels are identical in the method, it is prone to generate undesired ghosting artifacts, and this is shown in Section 3. In GSCM, the SNUC problem is translated into an optimization problem via minimizing the energy of the horizontal gradient of an image [18]. However, it is necessary to conduct “one-point” correction before other steps [21]. Furthermore, it exhibits unsatisfactory de-striping results. Additionally, it is unsuitable for real-time applications since complex computation is unavoidable in the algorithm.

A method based on column-wise midway histogram equalization (MHE) [6], [20] as proposed by Tendero et al. is an effective stripe removal method. Although the methods do not require the detection of the image edge and eliminate stripe noise without blurring the image details, they do not completely remove stripes and easily produce undesired image artifacts. Recently, Zhao et al. developed a scene-based SNUC method [18] to search for an optimal image in which the vertical gradient is close to the original image to the maximum possible extent and the energy of the horizontal gradient is minimized to the maximum possible extent. Though it exhibits excellent corrected results without introducing any image artifacts, the involved convolution integral calculation and flourier transformation make the real-time hardware implementation difficult. Additionally, an SNUC method based on single infrared image is proposed [21]. First, the method corrects the non-uniformity of the image to obtain a pilot image via an improved moment matching algorithm based on the means of gradient absolute values in each column (GAMC), and the local weighted-histogram specification algorithm (LWHS) is subsequently adopted to eliminate the stripe effect in the pilot image to obtain the final output image. The algorithm eliminates stripe noise effectually from infrared images although it easily produces undesired image artifacts on the image edge. In order to accomplish the goal of effectively compensating stripe-type spatial non-uniformity without blurring the image edge, Cao et al. developed an effective stripe noise removal method based on 1D guided filtering (1DGF) [22]. The method consists of two consecutive processing steps: performing horizontal edge-preserving image smoothing with 1D row-guided filtering and decomposing stripe noise from the overall extracted horizontal high-frequency signals with 1D column-guided filtering. They indicated that the algorithm efficaciously removed stripe noise as well as preserved the image edge. However, it is only suitable for low-textured infrared images. With respect to images with evident stripe texture information, it can lead to potential loss by blurring image details.

In 2016, Wang proposed a stripe noise removal method based on the linear observation model (MDC) [23] that is assumed as better than MHE. The key idea is that the differences between columns are minimized to subtract the estimated stripe noise from the original image to obtain a better result. However, they use fixed bias parameter for each column, and thus, similar ghosting artifacts or other undesired artifacts appear in the output image such as in MMSE. Additionally, as indicated by the latest studies, an improved stripe-denoising method that uses a new define image statistic measurement named 1D horizontal differential statistics (1DHDS) [24] was proposed by Cao et al. in 2017. The first processing step is identical to that in 1DGF to obtain a smoothed image and a high contrast part, and the image structure map is subsequently obtained through 1D-HDS computation. Finally, the stripe noise is estimated and subtracted from the original image via spatially adaptive filtering. The method is assumed as an improvement over those in a previous study on 1DGF and other comparative methods used on both column FPN elimination and image details preservation. Nevertheless, a global model is used in spatially adaptive filtering step, and thus, the bias parameters for column are fixed, and similar ghosting artifacts or other undesired artifacts, such as those in SNUC-MMSE and SNUC-MDC, can also appear.

Typically, SNUC is a subsequent step in the NUC processing [25]. The preliminary NUC steps are not ideal, and thus the statistic characteristics of stripe non-uniformity are broken. Therefore, the correction parameters in the same column are not identical and exhibit a few disturbances. Specifically, the difference in the local stripe non-uniformity in the same column is small after early NUC during image processing. Thus, stripe non-uniformity of adjacent pixels is more relevant than that of non-adjacent ones. Hence, the correction parameters of each column in a local region are close to each other. Several existing models including SNUC-MMSE, GAMC&LWHS, and SNUC-1DGF do not fully consider the disturbances and do not obtain good results or introduce image artifacts. In order to overcome the defects of existing methods, in the present study, a SNUC algorithm based on local spatial correlation is proposed to improve the results of NUC. In the algorithm, given changes in the pixel position and image scene, the correction parameters of the detector unit exhibit small variances when compared to that in the adjacent one above within a column. Thus, increasingly accurate correction parameters are obtained to eliminate the stripe noise without producing image blurring. Additionally, a strong target detection is conducted before the correction to avoid image artifacts.

Section snippets

Methodology

The correction models of several SNUC techniques assume that the responses of detector units are linear. Hence, the same gain coefficient and offset parameters are applied to the pixels of each column in NUC processing. With respect to an infrared image with M rows and N columns, the relation between the input x(i, j) and output y(i, j) images is as follows [26], [27], [28]: y(i,j)=g(j)x(i,j)+o(j).where i denotes the row coordinate, j denotes the column coordinate, and g(j) and o(j) correspond

Experimental results and analysis

In this section, the performance of the proposed algorithm is assessed by real infrared images acquired by a long-wavelength uncooled IRFPA and is further compared to several comparative algorithms. The infrared camera used in the experiments is ULIS PICO384P sensor with a resolution of 288 × 384 and a 17-μm pitch. The lens focal length is 19 mm, and F# is 1.0. The camera operates at temperatures ranging from 40 to 60 °C. The key parameters of the proposed algorithm used in the experiments are

Conclusion

In the study, a single-frame SNUC technique based on spatial correlation was presented to eliminate stripe noise from infrared images in real time. Given the influence of the previous NUC steps on the correction parameters, the proposed method did not use equal gain and offset for the pixels in the same column although it calculated the correction parameters for each pixel individually. Additionally, in order to protect the details of the texture information effectively, an effective method of

Acknowledgment

This research is supported by the Aero Science Foundation of China Project 20170179001.

References (37)

  • RiouO. et al.

    Nonuniformity correction and thermal drift compensation of thermal infrared camera

    Proc. SPIE

    (2004)
  • B. Cairnduff, Correction of non-uniformity of response in sensor arrays, G.B. Patent 2 445 254 A...
  • L. Huang, X. Sun, Non-uniformity correction method involves counting correlation coefficient according to current...
  • NugentP.W. et al.

    Correcting for focal-plane-array temperature dependence in microbolometer infrared cameras lacking thermal stabilization

    Opt. Eng.

    (2013)
  • CaoY. et al.

    Shutterless solution for simultaneous focal plane array temperature estimation and nonuniformity correction in uncooled long-wave infrared camera

    Appl. Opt.

    (2013)
  • RenJ. et al.

    Strip nonuniformity correction based on registration for infrared-focal plane arrays

    J. Infrared Millim. Waves.

    (2011)
  • HeZ. et al.

    Single image based non-uniformity correction of uncooled long-wave infrared detectors: A deep learning approach

    Appl. Opt.

    (2018)
  • TenderoY. et al.

    Efficient single image non-uniformity correction algorithm

    Proc. SPIE

    (2010)
  • Cited by (3)

    • An infrared image denoising model with unidirectional gradient and sparsity constraint on biomedical images

      2022, Infrared Physics and Technology
      Citation Excerpt :

      The global and local image features are combined to represent the rich structures in infrared cloud images. For the APB method, the IR biomedical image is analyzed by the frequency transformation [12–14]. The distribution of the transformed coefficients is fitted by the Gaussian or other function.

    • Research of Non Uniformity Correction for Short-wave Infrared Camera

      2022, IEEE International Conference on Knowledge Engineering and Communication Systems, ICKES 2022
    View full text