Flash scene video coding using weighted prediction

Abstract

A novel algorithm for coding flash scenes is proposed. In principle, flash scenes can be detected by analyzing the histogram differences between frames. The proposed algorithm then suggests an adaptive coding order technique for increasing the efficiency of video coding by taking account of characteristics of flash scenes in video contents. The use of adaptive coding technique also benefits to enhance the accuracy of derived motion vectors for determination of weighting parameter sets. Experimental results show that a significant improvement of coding performance in terms of bitrate and PSNR can be achieved in comparison with the conventional weighted prediction algorithms.

Introduction

Changes in illumination caused by a flash being fired during a press conference, a sport match, a news interview, etc., result in huge intensity difference between frames, which may easily be mistaken for motion estimation and compensation in video coding. Weighted prediction (WP) is one of the new tools in H.264 for coding scenes with brightness or illumination variations [1], [2], [3], [4], [5], [6]. With the use of WP, a reference frame f_ref is scaled and shifted by a WP parameter set including a multiplicative weighting factor W and an additive offset O, and then the sum of absolute difference (SAD) between the current frame f_c and the reference frame f_ref is defined as

$$\text{SAD}=\sum |f_c-(W\times f_{\mathit{ref}}+O)|$$

With the appropriate W and O, SAD becomes smaller so that motion estimation (ME) can obtain a well-matching block in the reference frame. It results in better coding efficiency. However, WP in H.264 is a frame-based approach which can only code the scenes with global brightness variations such as fade-in and fade-out efficiently [1], [2], [3], [4], [5], [6], [7], [8] but not the scenes with local brightness variations including flashlight (FL) scenes. Firing of flashes in a scene can cause the non-uniform intensity change distributed over the entire picture. It is therefore very difficult to find an accurate frame-based model to estimate the change of the intensity within the picture. Macroblock (MB)-based approaches [9], [10] in which different MBs in the same frame can use different W and O were proposed to solve the problem of local brightness variations. Unfortunately, this may lead to increased computational complexity, considering that it would be necessary to perform ME using all possible sets of W and O [9]. This MB-based concept was extended to multi-view video to solve the illumination and focus mismatches across views. In [11], a two-pass search algorithm was proposed. In the first pass, it uses mean-removed search to compute W and O for each MB candidate in the search window to find the disparity vectors. Based on the disparity vectors, depth levels are found and new filtered reference frames are generated for the second mean-removed search to find the best match for each MB. This two-pass algorithm requires high computational complexity and is only well suited to multi-view video coding. In [12], a human vision system based scheme was designed to solve the problems of coding FL scenes by interpolating and inpainting non-FL frames as FL frames. Nevertheless, the objective quality of FL frames is dropped a lot. In this paper, a MB-based scheme is specifically designed for coding FL frames.