GOP-level transmission distortion modeling for mobile streaming video

Abstract

Unequal loss protection is an effective tool in delivering compressed video streaming over packet-switched networks robustly. A critical component in any unequal-loss-protection scheme is a metric for evaluating the importance of different frames in a Group-Of-Pictures (GOP). In the case of video streaming over 3G mobile networks, packet loss usually corresponds to whole-frame loss due to low bandwidth and small picture size, which results in high error rates and thus most of the existing low-complexity transmission-distortion-estimate models may be ineffective. In this paper, we firstly develop a recursive algorithm to compute the GOP-level transmission distortion at pixel-level precision using pre-computed video information. Based on the study on the propagating behavior of the whole-frame-loss transmission distortion, we then propose a piecewise linear-fitting approach to achieve low-complexity transmission distortion modeling. The simulation results demonstrate that the proposed two models are accurate and robust. The proposed transmission distortion models are fast and accurate importance assessment tools in allocating limited channel resources optimally for the mobile streaming video.

Introduction

Nowadays, the expanded bandwidth for the air interfaces has made a solid ground for streaming media applications on 3G mobile network. With the advantages of wireless system in time and place, mobile streaming media service is very attractive. Due to different kinds of fading and multipath interference for the wireless channels, mobile video communication over 3G networks experiences burst packet losses. Moreover, the compressed video signal is extremely vulnerable against transmission errors, since low bit-rate video coding schemes rely on interframe coding for high coding efficiency. The coding structure of motion compensated interframe prediction creates strong spatio-temporal dependency in video frames [1], [2]. Consequently, the unavoidable packet losses during wireless transmission may result in error propagation of reconstructed video and thus induce severe quality degradation. This type of picture distortion is called transmission distortion.

In order to combat the effect of losses, many error control techniques are proposed for the packet video transmission [2], [3], [4], [5]. In Ref. [4], the error-control mechanisms are classified into four categories: forward error correction (FEC), retransmission, error resilience, and error concealment. For the streaming media applications, retransmission-based schemes are not allowed usually for real-time applications because of long time delay and hundreds of multicast members. Error-resilient schemes deal with packet loss on the compression layer, and most of them (e.g., resynchronization marking, data partitioning, and data recovery, etc.), are targeted to recover the decoding from bit errors. For packet video, transmission error is usually the entire packet loss and these bit-recovery based error-control mechanisms may cease to be effective [4]. Since complex error-concealment mechanisms have to be restricted by both the limited processing ability and power consumption of the handheld devices, the simplest and most common approach, previous frame repetition, is adopted in the mobile streaming media applications usually. FEC-based unequal loss protection (ULP) is one of the efficient error-control schemes used for the transmission of compressed video streaming over packet-loss networks [6], [7], by which limited channel resources are allocated efficiently to achieve increased rate-distortion (R-D) performance.

One of the most critical aspects of efficient resource allocation is accurate evaluation of the end-to-end video quality [8]. In the accurate transmission-distortion-estimate (TDE) schemes with moderate complexity, such as ROPE algorithm [9] and the statistics-based analytic model [10], the overall distortion accumulated from previous frames is computed to determine the coding mode for current macroblock (MB), where the total transmission distortion include distortion in current MB and its propagation in subsequent frames cannot be obtained. The low-complexity models include the method considering the intra refreshing and spatial loop filtering [11] and the method using the basic concepts of control systems [12]. It is worth noting that the low-complexity estimate models above are applicable for the low error-rates applications. For example, the error rate in Ref. [11] is less than 6% and that in Ref. [12] is about 8%. In the mobile video applications, packet loss usually corresponds to whole-frame loss due to low bandwidth and small picture size [13], which results in high error rates (an amount of blocks with nonzero motion vectors (MVs) are corrupted due to previous frame concealment) and thus the above low-complexity schemes are ineffective [11]. The simplified distortion-estimate model without considering the picture complexity, such as ELEP in Ref. [6] where only temporal error propagation is taken into account, are usually not accurate enough and thus optimal R-D performance cannot be achieved. Based on statistics of error propagation, a method for lightweight prediction of video distortion is proposed in Ref. [14].

In this paper, by exploiting the pre-computed information (such as MVs) of stored video signals, we first present a recursive TDE algorithm in this paper to compute the GOP-level transmission distortion for whole-frame losses. We then develop a low-complexity TDE model using piecewise linear-fitting approach based on the study on the error propagation behaviors of whole-frame losses. The experimental results demonstrate that the two proposed TDE models are accurate and robust. The proposed transmission distortion models provide a type of fast and accurate importance-assessment tools in allocating limited channel resources effectively.

The rest of this paper is organized as follows. In Section 2, we analyze the error propagation characteristics and unequal importance for different frames in a Group-Of-Pictures (GOP). Section 3 presents the recursive TDE algorithm for the stored video streaming, and Section 4 gives a piecewise linear-fitting approach based low-complexity estimate model. Simulation results are shown in Section 5. Finally, conclusions are drawn in Section 6.