Dictionary learning based reconstruction for distributed compressed video sensing

Highlights

• Leveraging more realistic video signal models that go beyond simple sparsity.

• A novel undersampling correlation noise model for subsampled video signals.

• To learn a dictionary that efficiently describes the video contents and structures.

• A maximum-likelihood (ML) dictionary learning based reconstruction for DCVS.

• Signal recovery is performed within ML learning, not as an independent task.

Abstract

Distributed compressed video sensing (DCVS) is a framework that integrates both compressed sensing and distributed video coding characteristics to achieve a low-complexity video coding. However, how to design an efficient reconstruction by leveraging more realistic signal models that go beyond simple sparsity is still an open challenge. In this paper, we propose a novel “undersampled” correlation noise model to describe compressively sampled video signals, and present a maximum-likelihood dictionary learning based reconstruction algorithm for DCVS, in which both the correlation and sparsity constraints are included in a new probabilistic model. Moreover, the signal recovery in our algorithm is performed during the process of dictionary learning, instead of being employed as an independent task. Experimental results show that our proposal compares favorably with other existing methods, with 0.1–3.5 dB improvements in the average PSNR, and a 2–9 dB gain for non-key frames when key frames are subsampled at an increased rate.

Introduction

Distributed video coding (DVC) [1] refers to a special video coding paradigm that encodes frames of a video sequence independently and decodes them jointly. As the temporal redundancies are exploited by the decoder exclusively, the computational burden is shifted from the encoder to the decoder, which makes DVC potentially applicable to many fields, e.g., wireless multimedia sensor networks (WMSN), video conferencing with mobile devices and surveillance systems. However, it still requires enormous data collection followed by data compression and thus, wastes valuable resources. Compressed sensing (CS) [2], [3], [4] is an innovative concept that has attracted considerable research interest in the signal processing community. It provides a new way to collect data incorporating both acquisition and compression, and consequently helps reduce the required number of measurements and transcend hardware limitations. Hence, the advantage of CS makes it a natural fit for DVC, due to the great reduction of sampling rate, power consumption and computational complexity.

Benefit from CS and DVC, distributed compressed video sensing (DCVS) [5], [6], [7], [8], [9], [10], [11] has recently emerged as a new way to directly capture video data via random projections at a low-complexity encoder, while performing joint reconstruction at a more complex decoder. The main challenge of DCVS is how to utilize the spatial/temporal redundancy in video at the decoder to achieve sparse representation and efficient reconstruction. One of the earlier works addressing DCVS was presented by Prades-Nebot et al. [5], in which a video sequence is divided into key frames and non-key (NK) frames. Key frames are intra encoded and decoded using traditional video compression standards; while NK frames are projected and recovered using CS techniques, with an adaptive redundant dictionary built by picking blocks from previously reconstructed frames. A similar method was proposed in [6], introduced as an inter-frame sparsity model. However, in these schemes, it is still required to capture huge amounts of raw video data for key frames, which are encoded using conventional compression algorithms.

Another DCVS framework was proposed in [7], [8], wherein the dictionary learning algorithm K-SVD [12] is directly employed by extracting samples from previous recovered frames together with the side information. As soon as the trained dictionary is obtained, NK frames are reconstructed by using the conventional sparse recovery algorithms. In this method, sparse representation and reconstruction are designed as independent tasks. However, this has a negative impact in terms of consuming resources, as the sparse coefficient calculation has already been included in the process of dictionary learning. Besides, a scalable framework of DCVS was presented in [9] to achieve optimal quality of service. In [10], an initialization and several stopping criteria were proposed for NK frames to speed up the convex optimization, and in [11] a measurement compression scheme by using the channel coding was proposed. Note that there also exist other literatures about CS-based video coding [13], [14], [15], [16], [17], e.g., a new dictionary generation scheme using an iterative fashion between reconstructing and filtering [15] and an adaptive-ADMM algorithm for CS with partial known support and signal value information [17] were proposed in our previous work. Nevertheless, most of these techniques, which are aimed to explore temporal/spatial redundancy at the encoder and achieve higher sampling efficiency, are not suited for DVC as far as limited resource is concerned.

In this paper, we propose a dictionary learning based reconstruction algorithm for DCVS. Our goal is to improve the reconstruction performance by leveraging more realistic signal models that go beyond simple sparsity and compressibility (by including the video signal structure), while retaining very low computation complexity at the encoder. One of our contributions is to introduce a novel correlation noise model (CNM) between the original video frame and its side information (SI) when video sequences are compressively sampled at a rate that is far below the Nyquist rate. To distinguish from the conventional notation in standard DVC, we denote our model as the “undersampled” CNM. To be specific, a new statistical model is presented in this work to characterize the error pattern of the correlation noise, and then offers an efficient way to describe the temporal correlation in undersampled videos. Another main contribution of this paper is that we propose a dictionary learning based reconstruction scheme, wherein we try to learn a dictionary that efficiently describes the content of video frames, and simultaneously permits to capture the correlation in sequences by including the CNM constraint. In this respect, we concentrate on the problem of two views and develop a maximum likelihood (ML) method. In our algorithm, the ML optimization is cast as an energy minimization problem, which can then be solved by iterating reconstruction and dictionary update. Consequently, our recovery method can achieve an efficient sparse representation for DCVS, and at the same time obtain the corresponding coefficients to recover video signals. In other words, both the dictionary learning and reconstruction are performed under the correlation constraint in order to achieve a good visual quality. To the best of our knowledge, there is no literature available to analyze CNM when the video sequence is compressively sampled, or to formulate the dictionary learning for DCVS with the prior on CNM.

Lastly, it is worth noting that in this paper we mainly focus on developing a dictionary learning based reconstruction algorithm for DCVS, which provides a novel fully low-complexity video compression paradigm and an alternative scheme adaptive to the environment where raw video data is not available, instead of competing compression performance against the current compression standards or DVC schemes, which need raw data available for encoding.

The rest of this paper is organized as follows. The overview of background is given in Section 2. The proposed ML dictionary learning method is described in Section 3. Section 4 presents the DCVS reconstruction with dictionary learning. Simulation results are described in Section 5, followed by conclusions in Section 6.