Spatial error concealment: A novel exemplar-based approach using segmentation

Abstract

In this paper, the problem of spatial error concealment for real-time applications is addressed. The proposed method can be categorized in exemplar-based error concealment approaches. In this category, a patch of corrupted pixels are replaced by another patch of the image that contains correct pixels. For splitting the erroneous block to different patches, a novel context-dependent exemplar-based algorithm based on a previously proposed segmentation method is proposed. The capability of the proposed method for concealment in diverse image regions is depicted. Our detailed conducted experiments show that the proposed method outperforms the state-of-the-art spatial error concealment methods in terms of output quality.

Introduction

Over the last decades, great improvements have been made in video communication by a growing demand for efficient video compression. Bandwidth limitations and the huge amount of data in raw image sequences were the very first problems in this type of communication. Therefore, a wide range of video compression methods was proposed for addressing this problem. Among different video compression methods, block-based approaches have been exploited extensively in video coding standards. In block-based approaches, a video frame is first split to nonoverlapping blocks and then processed block-wise. Compressed blocks are then transmitted to the receiver through an error prone network (Internet, wireless, etc.) and decompressed to reconstruct the original data. During data transmission through the network, the transmitted bitstream might be affected by different types of error. Two possible error control methods have been studied in the literature for recovering erroneous blocks in the receiver.

The first error control method is based on informing the transmitter and requesting for retransmission of the entire (or a part) of the erroneous frame [1]. This method suffers from the imposed retransmission delay that might be impractical in real-time applications. The second method performs by recovering the corrupted block using correctly received data. In the latter method, two general approaches have been proposed in the literature; error resilient and error concealment. Error resilient methods add some redundant information to the original data in the transmitter side and employ them in the receive side when error occurs [2], [3], [4], [5]. For better resiliency, more added redundant data is needed that is in contrast with compression purposes. Error concealment methods, on the other hand, employ the inherent spatial and temporal redundancy of the received data to conceal the error [6], [7]. In contrast with retransmission-based and resilient-based methods, these methods add no delay and no redundant data. However, they might suffer from higher computational complexity at the receiver side and less reconstructed image quality.

In this paper, a novel spatial error concealment method is presented that outperforms the state-of-the-art approaches in terms of reconstructed image quality and performs almost in real-time. The rest of this paper is organized as follows. In Section 2, previous works on error concealment are shortly introduced. The focus of this section is on spatial methods. The proposed method is presented in detail in Section 3. The conducted experimental results in diverse situations and in comparison with other existing methods are presented in Section 4. The time complexity of the method is also presented in this section. In Section 5, conclusion and future work are drawn.