An error resilient coding scheme for H.263 video transmission based on data embedding

Abstract

For entropy-coded H.263 video frames, a transmission error in a codeword will not only affect the underlying codeword but may also affect subsequent codewords, resulting in a great degradation of the received video frames. In this study, an error resilient coding scheme for real-time H.263 video transmission is proposed. The objective of the proposed scheme is to recover high-quality H.263 video frames from the corresponding corrupted video frames.

At the encoder, for an H.263 intra-coded I frame, the important data (the codebook index) for each macroblock (MB) are extracted and embedded into another MB within the I frame by the proposed data embedding scheme for I frames. For an H.263 inter-coded P frame, the important data (the coding mode and motion vector information) for each group of blocks (GOB) are extracted and embedded into the next frame by the proposed MB-interleaving GOB-based data embedding scheme. At the decoder, after all the corrupted MBs within an H.263 video frame are detected and located, if the important data for a corrupted MB can be extracted correctly, the extracted important data will facilitate the employed error concealment scheme to conceal the corrupted MB. Otherwise, the employed error concealment scheme is simply used to conceal the corrupted MB. As compared with some recent error resilient approaches, in this study the important data selection mechanism for different types of MBs, the detailed data embedding mechanism, and the error detection and concealment scheme performed at the decoder are well developed to design an integrated error resilient coding scheme. Based on the simulation results obtained in this study, the performance of the proposed scheme is better than those of four existing approaches used for comparison. The proposed scheme can recover high-quality H.263 video frames from the corresponding corrupted video frames up to a video packet loss rate of 30% in real time.

Introduction

To reduce transmission bit rate or storage capacity, many image/video compression techniques and standards have been developed for various applications, such as videophones, videoconferencing, interactive TV, multimedia databases, and the World Wide Web [1], [2], [3]. Transmitting real-time compressed images/video over noisy channels, such as the Internet or wireless networks, has become a challenging problem [1], [2], [3]. Transmission of real-time video usually has bandwidth, delay, and loss requirements. First, to achieve acceptable presentation quality, transmission of real-time video usually has a minimum bandwidth requirement. However, several current transmission channels, such as the Internet, cannot provide sufficient bandwidth reservation to meet such a requirement. Second, real-time video requires a bounded end-to-end delay, i.e., each video packet must be received at the decoder in time to be decoded and displayed. Third, loss of packets (transmission errors) can potentially degrade the visual quality to human eyes. However, several current transmission channels, such as the Internet, cannot support any quality of service guarantee to meet the above three requirements. Hence, to achieve robust real-time video transmission, two kinds of issues, namely, congestion control and error control, should be addressed. The main purpose of congestion control is to prevent packet loss. However, packet loss is usually unavoidable. Hence, error control is required to maximize video presentation quality in the presence of packet loss (transmission errors) [1].

For entropy-coded H.263 video frames [4], [5], [6], a transmission error in a codeword will not only affect the underlying codeword but may also affect subsequent codewords, resulting in a great degradation of the received video frames. To cope with the synchronization problem, each of the top two layers of the H.263 hierarchical structure [4], [5], [6], namely, picture and group of blocks (GOB), is ahead with a fixed-length start code (a synchronization codeword). After the decoder receives a start code, the decoder resynchronizes regardless of the preceding slippage. However, a transmission error may affect the underlying codeword and its subsequent codewords within the corrupted GOB. Moreover, because of the use of motion-compensated interframe coding, the effect of a transmission error may be propagated to the subsequent video frames, as illustrated in the example shown in Fig. 1. In this study, an error resilient coding scheme for real-time H.263 video transmission is proposed.

In general, error resilient approaches include three categories [1], [2], [3], namely: (1) the error resilient encoding approach [7], [8], [9], [10], [11], [12], [13], [14], (2) the error concealment approach [15], [16], [17], [18], [19], [20], [21], and (3) the encoder–decoder interactive error control approach [22]. The error resilient encoding approach can be further divided into four categories, namely, (1) the robust entropy coding approach [7], [8], (2) the error resilient prediction approach [9], [10], (3) the layered coding with unequal error protection approach [11], [12], and (4) the multiple description coding approach [13], [14].

The robust entropy coding approach [7], [8] copes with the synchronization problem, whereas the error resilient prediction approach [9], [10] copes with the temporal error propagation problem. The layered coding with unequal error protection approach [11], [12] divides an image/video bitstream into a base layer and one or several enhancement layer(s) with unequal degrees of error protection. The multiple description coding approach [13], [14] divides an image/video bitstream into several sub-bitstreams, known as descriptions. Any single description can provide a basic quality, and more descriptions together will provide an improved quality. The error concealment approach [15], [16], [17], [18], [19], [20], [21] conceals the corrupted (lost) information due to transmission errors in a received video bitstream at the decoder by using (1) spatial (spectral) [15], [16], (2) temporal [17], [18], or (3) hybrid (spatial and temporal) [19], [20], [21] image/video information. On the other hand, if a feedback channel can be set up from the decoder to the encoder [22], the decoder can inform the encoder about which parts of the transmitted information are corrupted, and then the encoder can adjust its encoding operations accordingly to suppress or eliminate the effect of transmission errors. If the automatic repeat request function is supported, the corrupted (lost) packets can be retransmitted with some delay constraint.

Using the information from the spatially and/or temporally neighboring blocks of a corrupted block to conceal the corrupted block may introduce some problems. First, the information from the spatially and/or temporally neighboring blocks may not be available (may also be corrupted). Second, the video contents between a corrupted block and its spatially and/or temporally neighboring blocks may be very different. In these cases, the concealed results of the foregoing error concealment approaches are usually not good enough. Recently, several error resilient coding approaches based on data embedding have been proposed [23], [24], [25], [26], [27], [28], [29], in which some important data useful for error concealment performed at the decoder can be embedded into the video frames when these video frames are encoded at the encoder. The embedded data should be “almost” invisible and cannot degrade the quality of the video frames greatly, just like digital watermarking [30]. At the decoder, if some corrupted blocks are detected and located, the important (embedded) data for the corrupted blocks will be extracted and used to facilitate error concealment performed at the decoder. Song and Liu [26] proposed a data embedding scheme for error-prone channels, in which some redundant information used to protect motion vectors and coding modes of MBs in one frame is embedded into the motion vectors in the next frame. Based on the assumption that the next video frame of a corrupted video frame is correctly received, the decoder can exactly recover the motion vectors of the corrupted GOBs in the corrupted frame. Yilmaz and Alatan [27] proposed an error resilient video transmission codec utilizing imperceptible embedded information for error detection, resynchronization, and reconstruction. For an intra-coded frame, both a spatial error recovery technique embedding the edge orientation information of each block and a resynchronization technique embedding the bit-length of each block are proposed to recover the corrupted blocks. For an inter-coded frame, the embedded motion vector information is used to recover the corrupted blocks. In some recent error resilient approaches [26], [27], [28], the case where an MB (or block) and its important data (embedded into another MB or block) are corrupted simultaneously is not well handled. In this case, the embedded data for a corrupted MB (or block) are not available and an error concealment scheme is required. In this study, the important data selection mechanism for different types of MBs, the detailed data embedding mechanism, and the error detection and concealment scheme performed at the decoder are well developed to design an integrated error resilient coding scheme. In addition, the above-mentioned case will be well handled.

In this study, at the encoder, for an H.263 intra-coded I frame, the important data (the codebook index) for each MB are extracted and embedded into another MB within the I frame by the proposed data embedding scheme for I frames. For an H.263 inter-coded P frame, the important data (the coding mode and motion vector information) for each GOB are extracted and embedded into the next frame by the proposed MB-interleaving GOB-based data embedding scheme. At the decoder, after all the corrupted MBs within an H.263 video frame are detected and located, if the important data for a corrupted MB can be extracted correctly, the extracted important data will facilitate the employed error concealment scheme to conceal the corrupted MB. Otherwise, the employed error concealment scheme is simply used to conceal the corrupted MB.

This paper is organized as follows. The employed error detection and concealment scheme for H.263 video transmission is given in Section 2. The proposed error resilient coding scheme for real-time H.263 video transmission based on data embedding is addressed in Section 3. Simulation results are included in Section 4, followed by concluding remarks.