Velocity filtering for attenuating moving artifacts in videos using an ultra-low complexity 3-D linear-phase IIR filter

Abstract

An ultra-low complexity three-dimensional (3-D) linear-phase infinite-extent impulse response (IIR) velocity filter is proposed for attenuating moving artifacts in videos while enhancing objects moving on smooth approximately-linear trajectories. The proposed 3-D linear-phase IIR velocity filter consists of an ultra-low complexity 3-D wide-angle linear-phase IIR cone filter bank between two two-dimensional (2-D) spatial variable-shift filters. The ultra-low complexity 3-D wide-angle linear-phase IIR cone filter bank consists of a one-dimensional (1-D) temporal modified discrete Fourier transform filter bank and 2-D spatial allpass, IIR highpass and allstop filters. A typical 3-D linear-phase IIR velocity filter of order $4\times 4\times 510$, applied to enhance a heavily corrupted test video signal, requires only 26 real multiplications and 60 real additions to process a sample and provides a scene-complexity-independent signal-to-interference-and-noise ratio improvement of 13.86 dB. The proposed 3-D linear-phase IIR velocity filter is employed to attenuate sunlight flicker patterns in shallow underwater videos, and experimental results are presented to confirm the effectiveness of the proposed method and its robustness to motion estimation errors.

Appendix: Proof of the near-perfect reconstruction of the 3-D wide-angle linear-phase IIR cone filter bank

The near-perfect reconstruction of the cone filter bank can be proved following an approach similar to that employed in Edussooriya et al. (2013). To this end, the input-output relationship of the cone filter bank is expressed, following (Kuenzle and Bruton 2006; Karp and Fliege 1999), as

$$\begin{aligned} \widehat{I}_{out}(z_x,z_y,z_t) = z_t^{-M/2}\sum _{l=0}^{M/2-1}A_l(z_x,z_y,z_t)\widehat{I}_{in}(z_x,z_y,z_tW_M^{2l}), \end{aligned}$$

(18)

where $\widehat{I}_{in}(z_x,z_y,z_t)$ and $\widehat{I}_{out}(z_x,z_y,z_t)$, $(z_x,z_y,z_t)\in \mathbb {C}^3$, are the 3-D $\mathscr {Z}$ transforms of the input signal $\widehat{i}_{in}(n_x,n_y,n_t)$ and the output signal $\widehat{i}_{out}(n_x,n_y,n_t)$ of the cone filter bank, respectively, and

$$\begin{aligned} A_l(z_x,z_y,z_t) = \frac{1}{M}\sum _{k=0}^{M-1}H_k(z_tW_M^{2l})G_k(z_x,z_y)F_k(z_t), \qquad l = 0,1,\ldots ,(M/2-1). \end{aligned}$$

(19)

The output signal $\widehat{I}_{out}(z_x,z_y,z_t)$, given in (18), consists of the desired alias-free component $D(z_x,z_y,z_t)$ corresponding to $l=0$ and given by

$$\begin{aligned} D(z_x,z_y,z_t) = \frac{z_t^{-M/2}}{M}\sum _{k=0}^{M-1}H_k(z_t)G_k(z_x,z_y)F_k(z_t)\widehat{I}_{in}(z_x,z_y,z_t), \end{aligned}$$

(20)

and the undesired aliased component $A(z_x,z_y,z_t)$ having $(M/2-1)$ aliasing terms corresponding to $l=1,2,\ldots ,(M/2-1)$ and given by

$$\begin{aligned} A(z_x,z_y,z_t)&= \frac{z_t^{-M/2}}{M}\sum _{l=1}^{M/2-1}\left( \sum _{k=0}^{M-1}H_k(z_tW_M^{2l})G_k(z_x,z_y)F_k(z_t)\right) \widehat{I}_{in}(z_x,z_y,z_tW_M^{2l}) \nonumber \\&= z_t^{-M/2}\sum _{l=1}^{M/2-1}A_l(z_x,z_y,z_t)\widehat{I}_{in}(z_x,z_y,z_tW_M^{2l}). \end{aligned}$$

(21)

In order to achieve the perfect reconstruction, the magnitude and the phase responses of the cone filter bank should be unity and linear, respectively, inside the passband. Moreover, the aliased component $A(z_x,z_y,z_t)$ should be zero inside the passband of the cone filter bank. In what follows, we show that the cone filter bank approximately satisfies the above three conditions and, consequently, achieves the near-perfect reconstruction. To this end, (19) is rewritten, by employing (6), as

$$\begin{aligned} A_l(z_x,z_y,z_t) = \frac{\,z_t^{-N_P}W_M^{-lN_P}}{M}\sum _{k=0}^{M-1}P(z_tW_M^{2l+k})G_k(z_x,z_y)P(z_tW_M^k). \end{aligned}$$

(22)

Because the prototype filter $P(z_t)$, designed in Sect. 4.1, has a stopband edge less than $2\pi /M$ and a fairly high stopband attenuation, it can be shown that (Lin and Vaidyanathan 1994)

(23)

By substituting (23) into (22), we can show that

$$\begin{aligned} A_l(z_x,z_y,z_t)\approx 0, \quad l=1,2,\ldots ,(M/2-1) \end{aligned}$$

(24)

and, therefore, $||A(z_x,z_y,z_t)||_2$ can be made arbitrarily small in relation to $||\widehat{I}_{in}(z_x,z_y,z_t)||_2$. In this case, the cone filter bank is almost alias free, and the transfer function $C(z_x,z_y,z_t)$ can be expressed as

$$\begin{aligned} C(z_x,z_y,z_t)&\approx \frac{z_t^{-M/2}}{M}\sum _{k=0}^{M-1}H_k(z_t)G_k(z_x,z_y)F_k(z_t) \nonumber \\&\approx \frac{z_t^{-(N_P+M/2)}}{M}\sum _{k=0}^{M-1}\left[ P(z_tW_M^k)\right] ^2G_k(z_x,z_y). \end{aligned}$$

(25)

Considering the fact that $G_k(z_x,z_y)=0$ for all the 2-D spatial allstop filters, (25) can be rewritten as

$$\begin{aligned} C(z_x,z_y,z_t) \approx \frac{z_t^{-(N_P+M/2)}}{M}\sum _{k\in \mathscr {C}}\left[ P(z_tW_M^k)\right] ^2G_k(z_x,z_y), \end{aligned}$$

(26)

where $\mathscr {C}=\{0,1,\ldots ,M_h,M-M_h,M-(M_h+1),\ldots ,M-1\}$. Because $P(\mathrm {e}^{\,j\omega _t})\approx \sqrt{M}$ and $G_k(\mathrm {e}^{\,j\omega _x},\mathrm {e}^{\,j\omega _y})\approx 1$, $k\in \mathscr {C}$, inside their respective passbands, the frequency response of the cone filter bank can be obtained from (26) as

$$\begin{aligned} C(\mathrm {e}^{\,j\omega _x},\mathrm {e}^{\,j\omega _y},\mathrm {e}^{\,j\omega _t}) \approx z_t^{-(N_P+M/2)}, \qquad (\omega _x,\omega _y,\omega _t)\in \mathscr {P}, \end{aligned}$$

(27)

where $\mathscr {P}$ denotes the region of support corresponding to the passband of the cone filter bank in the 3-D frequency domain. It is clear from (27) that the magnitude response $\left| C(\mathrm {e}^{\,j\omega _x},\mathrm {e}^{\,j\omega _y},\mathrm {e}^{\,j\omega _t})\right|$ of the cone filter bank is approximately unity inside the passband whereas the phase response $\angle C(\mathrm {e}^{\,j\omega _x},\mathrm {e}^{\,j\omega _y},\mathrm {e}^{\,j\omega _t})$ of the cone filter bank is approximately linear inside the passband. Therefore, the cone filter bank achieves the near-perfect reconstruction. $\square$