QoE-aware video streaming for SVC over multiuser MIMO–OFDM systems

Highlights

• We derive a novel QoE-aware video adaptation (QoEVA) scheme.

• We propose an efficient QoE-aware resource allocation (QoERA) scheme.

• We combine QoEVA and QoERA to design a general QoE-aware video streaming solution.

Abstract

We propose a QoE-aware video streaming solution to maximize multiuser QoE for Scalable Video Coding (SVC) streaming over multiuser (MU) Multiple-Input Multiple-Output (MIMO)–Orthogonal Frequency Division Multiplexing (OFDM) systems. We achieve it by integrating novel QoE-aware video adaptation (QoEVA) and QoE-aware resource allocation (QoERA) schemes. We first study QoEVA of SVC via a subjective video quality assessment database and derive QoE-optimized scalability adaptation tracks. A rate-QoE model is then constructed to approximate the track and is employed to design QoERA. By proving the NP-hardness of the QoERA problem, we propose an adaptive solution where resource block assignment, power allocation and modulation selection are jointly optimized to enhance multiuser QoE. Experimental results show that the proposed QoEVA significantly performs better than the conventional video adaptation schemes and the proposed QoERA achieves much better user experience when compared to state-of-the-art solutions. Our solution can be employed for both pre-coded and live video streaming.

Introduction

Nowadays, media service providers are keen on improving the users’ perception of quality, commonly referred to as the Quality of Experience (QoE) [1], in wireless video communication. User’s perception of streaming video characterized in terms of QoE is directly measured by the acceptability of the end users. QoE is more related to but differs from the extensively studied concept Quality of Service (QoS) [2], which mainly focuses on system related parameters such as bandwidth, delay, loss, etc. However, QoS lacks an important element, the human perception, in quantifying the perceived video quality [3]. It has been revealed that the structured approach to QoE can be designed from user and technical aspects [4], [5] and validated by mean opinion score (MOS) [6]. MOS evaluations tend to be computationally intensive, cumbersome, not repeatable, and are often hard to adapt to real time quality assessment. The well-known video quality metrics such as video quality metric (VQM) [7] and Structural Similarity Index (SSIM) [8] significantly improve over the traditional video quality metrics such as MSE and PSNR. They consider some inherent characteristics such as the forms of distortion and structural information to improve the confidence level of quality evaluation. However, they cannot reliably replace the subjective evaluation. Recently, there have been many efforts to study QoE modeling [9], [10], [11], [12], QoS to QoE mapping [13], [14], [15], etc. However, the design of the QoE-aware video streaming is still an open issue.

To enable QoE-aware wireless video streaming, Scalable Video Coding (SVC) [16] stands out with its graceful rate adaptation capabilities to cope with bandwidth scarcity and network variation. SVC is encoded with a most important base layer whilst quality refinement or resolution enhancement is realized by successive encoding of more enhancement layers. SVC offers spatial, temporal and quality scalabilities for enhancement in picture size, frame rate and picture fidelity, respectively. QoE-aware streaming can be achieved by excluding some enhancement layer packets during transmission to conform to network variation, hardware heterogeneity or users’ requirement.

To achieve this, we combine QoE-aware resource allocation (QoERA) with QoE-aware video adaptation (QoEVA). A QoE-aware SVC streaming solution aims to maximize video QoE under rate constraints:

$$\max \mathit{Q}(\mathit{R})$$

$$\text{s.t.}\mathit{R}(\mathit{d}\text{,}\mathit{t}\text{,}\mathit{q})⩽{\mathit{R}}_{\max }$$

$$(\mathit{d}\text{,}\mathit{t}\text{,}\mathit{q})\in \mathrm{\Xi }$$

where $\mathit{d}\text{,}\mathit{t}$ and $\mathit{q}$ are the spatial, temporal and quality layer IDs of the adapted bitstreams, respectively. The function (1a) defines the objective of QoERA and the rate-QoE model. The target adaptation rate constraint in (1b) relates to a candidate scalable layer which is selected from a triplet set $\mathbf{\Xi }$ in (1c). Rate constraints ${\mathit{R}}_{\max }$ is determined by QoERA under given network conditions. The bold letters here denote vectors for multiuser video streaming.

Existing work can be helpful for designing QoEVA but their limitations are in different aspects. As for (1a), the majority of work either lacks a QoE model [17], [18], [19] or employs a QoE-agnostic model [20], [21]. Some general conclusions have been drawn in [17], [18], [19] by comparing adapted videos in different scalability combinations but no explicit QoE model has been summarized. The default video adaptation schemes in Joint Scalable Video Model (JSVM) aim at maintaining a good PSNR for target video but PSNR cannot reflect the true QoE [20], [21]. For constraints (1b), (1c), some earlier attempts have studied QoE-aware adaptation but they focused only on partial scalability¹ [22], [23], [24], [25]. The authors in [22] revealed that users generally favor temporal scalability over quality scalability. While in [23], the authors found that for full-length movie, subjects may prefer quality adaptation. Another work in [24] investigated spatial and temporal scalabilities and the findings suggested that bandwidth savings should be achieved by reducing spatial size before decreasing frame rate. Moreover, we have set up a QoE database and derived QoE model for source adaptation in [26] but the scheme works on a whole video sequence, which is not valid for video streaming. The adaptation and QoE model should also be extended and validated for videos outside the database and for distorted videos received from network transmission. Nevertheless, existing work has introduced some insights and motivated us to propose a more accurate QoEVA scheme to facilitate QoERA.

QoEVA needs to be incorporated with resource allocation in multiuser video streaming to overcome the challenges of resource scarcity, user competence, delay, packet loss, etc. The majority of emerging wireless systems employ multiuser (MU) Multiple-Input Multiple-Output (MIMO)–Orthogonal Frequency Division Multiplexing (OFDM) techniques to increase system performance [27]. MU MIMO–OFDM enables efficient resource allocation in joint time, frequency, and spatial domains, which can significantly enhance the quality of video streaming. It is necessary to study the QoE issue in this system because of its wide deployment. Our objective is to design a QoE-aware resource allocation (QoERA) scheme in the joint spatial-time–frequency domain. We first recall some useful transmission mechanisms in existing work, which can be generally classified as QoS provision and prioritized delivery. The traditional round-robin resource allocation renders no QoS provision and hence, many adaptive schemes have recently been proposed. Among those representative work that has employed MU MIMO–OFDM systems, multicarrier maximum sum rate (MMSR) [28] aims to maximize throughput and adaptive radio resource allocation (ARRA) [29] distributes resource based on head-of-line (HOL) packet delay. In [30], the authors adopted a joint power allocation and bit loading (JPABL) scheme to maximize video PSNR. On the other hand, as video packets are encoded with dependency, it always observes the performance improvement by prioritized delivery. The authors have proposed to generate layered SVC bitstreams for prioritized delivery in [31], [32], [33], [34], [35], etc. Prioritized delivery has also been combined with link adaptation, traffic control, and other schemes to enhance video quality.

Although lessons can be learned from existing schemes, QoE-aware video streaming calls for more in-depth studies. From network perspective, QoS provision cannot guarantee QoE satisfaction [28], [29], [30]. From video perspective, existing schemes are generally not designed on QoE criterion or they define QoE simply as a function of rate [31], [32], [33], [34], [35], which is not satisfactory. QoS based schemes always neglect the intrinsic properties of video packets such as differentiated packet priority, bursty packet sizes and dependencies. For example, throughput maximization will make unfair allocation which will severely harm certain users’ visual quality. For existing prioritized delivery schemes, some consider QoE factors and some not. Even for the former, they only cover partial scalability of SVC. We extend our previous work [35] which applies our source adaptation model [26] for video streaming. The previous work [35] is not practical for real application as it carries out resource allocation on sequence level and inherently requires the same adaptation scheme for all video segments in a sequence. It forces to a high playback delay and also assumes that transmitted data can be perfectly received without loss, which is not real. Moreover, QoE is only related to transmission rate in [35]. A practical QoE-aware streaming solution should jointly consider rate, delay, jitter, packet loss, etc., which has not been covered by most of existing work. To address this issue, we present a QoE-aware solution for SVC video streaming over MU MIMO–OFDM systems. Our contributions are threefold:

• There are two time scales in QoE-aware video streaming. We propose a progressive video streaming framework to align them by gradual streaming of a video segment across multiple resource allocation units. In a video timescale we achieve QoEVA and in a network timescale we execute QoERA. The framework makes sure that resource allocation can be optimized at the video adaptation time scale and meanwhile, best utilizes the channel diversity generated at the channel variation time scale. We accommodate multiple factors, i.e., rate, delay, jitter, packet loss, which affect video QoE, in a comprehensive framework. The framework is applicable to both pre-coded and live video streaming.

• To study QoEVA, a subjective video quality assessment database is constructed. Scalability adaptation tracks² are summarized to provide useful guidelines to adapt SVC bitstream in various contents and to facilitate progressive video streaming. A rate-QoE model is proposed for designing our QoERA. We demonstrate the effectiveness and the application of the QoE model in both source adaptation and wireless streaming. The QoE modeling for wireless streaming provides a novel no-reference (NR) video quality assessment and management solution.

• To enable progressive streaming, a QoERA scheme is designed to maximize multiuser QoE for SVC video streaming over MU MIMO–OFDM systems. QoERA jointly considers the spatial multiplexed resource block assignment, power allocation and modulation selection to maximize the overall QoE. The proposed solution tackles an NP-hard problem in a skillful way and meanwhile, achieves significant better performance than conventional methods with highly reduced complexity.

The remainder of this paper is organized as follows. We introduce the background and generalize a novel QoE-aware video streaming framework in Section 2. We elaborate on the video quality assessment database, video adaptation schemes and rate-QoE model in Section 3. In Section 4, we present the QoE-aware resource allocation framework for SVC streaming over MU MIMO–OFDM systems. The experimental results and discussions are provided in Section 5. Finally, conclusions are given in Section 6.