Motion estimation and compensation from noisy image sequences: A new filtering scheme

https://doi.org/10.1016/j.imavis.2006.05.010Get rights and content

Abstract

Motion estimation and compensation are proposed to be used as a filtering scheme. This latter is perceived in the motion compensation stage, when noisy frames are predicted from a clean one in the same sequence. The prediction source frame can be obtained simply by filtering spatially one selected frame in the sequence. The filtering efficiency is under-constrained by the accuracy of the motion vector field estimated between the prediction source and the other noisy frames. A study has been conducted on the motion estimation from noisy image sequences showing the conditions under which the motion vector field accuracy is preserved. Simulation results have shown that a total decrease by about 80% in computation time is achieved compared to the classical motion-compensated filtering.

Introduction

During the last few years, applications utilizing digital video such as multimedia services, teleconferencing, surveillance, object tracking, medical and astronomical imaging, etc., are growing spectacularly. In conjunction with this increased use of digital video, the demand for noise filtering algorithms is also on the rise. The sources of noise that can corrupt an image sequence are numerous. Examples of the more prevalent ones include, the noise introduced by the camera, shot noise that originates in the electronic hardware and the thermal or channel noise [1]. Most noise sources are well modelled by additive white gaussian noise model.

Noise filters for video, which make use of both spatial and time correlations among pixel intensities, are in general called spatiotemporal or three-dimensional (3-D) filters. A special class of spatiotemporal filters is that of temporal filters where one-dimensional (1-D) filtering techniques in the temporal direction are applied. Approaches that attempt at taking full advantage of the time redundancy incorporate motion detection and compensation [2]. The main distinction between the two is that in the motion-compensated approach the effect of interframe motion is explicitly estimated, whereas in the motion-detection approach the effect of interframe is implicitly accounted for in the design of the filter. In motion-compensated filtering, first a motion estimation algorithm is applied to the noisy image sequence in a first stage to estimate the motion trajectories. The filtering is then performed along the motion trajectory using either FIR filter [3], [4] or an IIR filter [5], [6]. A systematic overview of 3-D and 1-D motion-compensated and nonmotion-compensated filters is given in [1].

In this paper, we propose a novel filtering approach, which is somewhat different from all the existing ones. The basic idea is that noise can be reduced from a noisy frame in a sequence only by performing motion estimation and compensation; that is the approach originality. It is well known that by performing efficient motion estimation between two adjacent frames, one could be predicted (motion-compensated) from the other using the motion vector field estimated between the both. If the frame source is not noisy or less affected by noise, the predicted frame is filtered. This filtering scheme is under-constrained by the accuracy of the motion vector field and the quality of the frame source of prediction. Hence, this paper is principally directed toward the study of the motion estimation and compensation process between frames in a noisy context.

The experiments conducted have shown that our filtering scheme so viewed outperforms the classical filtering approach in terms of computation complexity load for a comparable visual quality. This was expected because motion compensation (MC) so viewed performs at once a filtering contrarily to any motion-compensated filter. This alleviates greatly the computation load.

The paper is organised as follows: in Section 2, we first present briefly the block matching motion estimation algorithm and give some background information on the temporal frame prediction using motion compensation. Next, we present in-depth analyses and observations on the motion compensation procedure in a noisy context where we derive our filtering scheme. The experimental evaluation is given in Section 3. Section 4 summarizes the results of the paper and gives a conclusion.

Section snippets

Motion estimation and compensation: a new filtering scheme

Estimating motion from two or more consecutive image frames has become an important problem in image sequence processing and for which there are a variety of applications. There are a large number of techniques treating the problem of motion estimation in the literature [7], [8], [9], [10]. One popular technique is the Block Matching Algorithm (BMA) [9], [10]. In BMA, motion vectors between two frames are estimated by subdividing one frame into blocks of size T × T and by assuming that all pixels

Experimental evaluation

We have divided our experiments into two distinct test series: the first one investigates the effect of noise variance on the possibility of getting filtering from motion compensation (MC). The second part of experiments presents a performance comparison between the proposed filtering approach (MC) and a classical motion-compensated filter: the 3-D adaptive weighted averaging filter (3-D AWA) [2], [4]. The main features of the 3-D AWA are summarised below.

The test sequences used in simulations

Summary and conclusion

We have shown that motion estimation and compensation can be used as a filter. This new filtering scheme depends on two factors: the correctness of the estimated motion vector field and the quality of the frame to be the prediction source in the sequence. For the former factor, analyses and experiments have been done indicating that for image sequences corrupted with zero-mean white gaussian noise, motion estimation correctness is preserved. For the second factor, we dealt with the problem of

References (12)

  • J. Brailean et al.

    Noise reduction filters for dynamic image sequences: a review

    Proc. IEEE

    (1995)
  • N.B. Benmoussat, Etude et mise en œuvre de méthodes d’estimation de mouvement et de filtrage spatiotemporel de...
  • M.I. Sezan, M.K. Ozkan, S.V. Fogel, Temporally adaptive filtering of noisy image sequences using a robust motion...
  • M.K. Ozkan, M.I. Sezan, M. Tekalp, Adaptive motion-compensated filtering of noisy image sequences, in: IEEE...
  • A. Katsaggelos, R.P. Klheihorst, S.N. Efstratiadis, R.L. Lagendjik, Adaptive image sequences noise filtering methods,...
  • T.A. Reinen, Noise reduction in heart movies by motion compensated filtering, in: Proceedings of the SPIE Visual...
There are more references available in the full text version of this article.

Cited by (6)

  • Enhancing dynamic videos for surveillance and robotic applications: The robust bilateral and temporal filter

    2014, Signal Processing: Image Communication
    Citation Excerpt :

    The performance of this method for denoising color videos corrupted with Gaussian noise was superior comparative to other spatiotemporal filters, for example, the STGSM and the NLM. Finally, Benmoussat et al. [3] propose a filtering scheme for video sequences. The filtering is conducted by computing the optical flow (block matching algorithm) to compensate the motion as a temporal prediction step of previous frames (already denoised).

  • Digital video stabilization in static and dynamic scenes

    2015, Intelligent Systems Reference Library
  • Motion estimation for objects analysis and detection in videos

    2012, Intelligent Systems Reference Library
View full text