Analytical model for MPEG video frame loss rates and playback interruptions on packet networks

Abstract

This paper presents and studies objective video quality evaluation techniques for a network where frame losses can be considered independent, for example a best effort not heavy loaded packet switching network. The total or partial loss of a frame’s information affects the quality of video playback, as the frame cannot be decoded and other frames that depend on it cannot be correctly decoded too. Therefore, during some time the video playback has errors in the image and the user will perceive them as interruptions. In this paper, the total number of decoded frames and the video playback interruptions duration will be considered important parameters to quantify the video quality. The analytical formulation for them will be presented and the importance of considering them together will be highlighted.

Appendix: Cut lengths in a video

In a video flow that suffers frame losses, the cut length can vary from one frame length to the total length of the video. The cut length depends on the grouping of the directly lost frames and of the non-decodable frames due these lost frames. The cuts must be understood as experienced by the user, therefore they must be measured from the presentation order of the videos, not the transmission order where the losses take place.

For example (Fig. 6), in a video with the IPBBPBB GoP structure in transmission order, if the second B-frame and second P-frame are lost, the four last frames of the GoP could not been decoded. But in presentation order, two disconnect cuts of size 1 and 3 will be generated. This makes the analytical model of cuts more complex.

Fig. 6

Left: second B-frame and second P-frame in transmission order are lost; Center: the last B-frames cannot be decoded; Right: in presentation order two cuts of sizes 1 and 3 are generated

When an I-frame is directly lost the whole GoP cannot be decoded, creating a single cut of at least length N. In the case of an open GoP, the last B-frames from the previous GoP will be non-decodable and they will be part of the same single cut of length N + (M − 1) when considered in presentation order. If we define the control variable $z=\frac{N_B}{M-1}-N_P$, where z = 1 if the GoP is an open one, and z = 0 if it is a closed GoP, then the cut length will be N + z * (M − 1).

If the I-frame of the next GoP is lost too, all the frames from the next GoP will not be decoded and a single cut of length 2 * N + z * (M − 1) will be generated. In general, when j consecutive I-frames are lost, a single cut of length j * N + z * (M − 1) will be generated.

The loss of a P-frame makes impossible to decode all the following frames in the GoP. As all these frames are consecutive, both in transmission and presentation order, only one cut is generated. If the lost P-frame is the last or N _P th P-frame of the GoP, the cut will be of M + z * (M − 1) frames length. If the lost P-frame is the penultimate or (N _P  − 1)th P-frame of the GoP, the cut will be of 2 * M + z * (M − 1) frames length. If the lost P-frame is the first P-frame of the GoP, the cut will be of N _P * M + z * (M − 1) frames length. In general, when the (N _P  + 1 − i)th P-frame of the GoP is lost, a single cut of i * M + z * (M − 1) will be generated.

If the I-frame from the next GoP is lost too, then all the frames from the next GoP will not be decoded. Again, all these frames are consecutive and result in only one cut of N + i * M + z * (M − 1) frames. In general, when the (N _P  + 1 − i)th P-frame of the GoP is lost and the next j I-frames are lost, a single cut of length j * N + i * M + z * (M − 1) will be generated.

The loss of c consecutive B-frames of a B-frames block does not affect other frames and it creates only one cut. If the last B-frame of the B-frames block is lost and the next frame (either I-frame or P-frame) is lost too, in presentation order the non-decoded frames will not be consecutive. The B-frames will be on one cut and the frames non-decoded from the lost I- or P-frame will be on the other cut.

The possible cut lengths are summarized in (19), where F is the number of frames in the video.

$$ \begin{array}{rll} \label{eq:c2} c &=& [ 1 \ldots M-1 ,\: j * N + i * M + z * (M-1) ,\: (j+1) * N + z * (M-1) ] \\ i &=& 1 \ldots N_P \\ j&=& 0 \ldots N_G-1 \end{array} $$

(19)

A similar approach was used for the analytical expression of Q in [29]. On the following we obtain the analytical expression of N _cut[c], the number of cuts of c frames length.

P _{{I, P, B}} is the probability of a {I, P, B}-frame being directly lost. It is assumed that direct frame losses are mutually independent.

A cut of length (j + 1) * N + z * (M − 1) frames will be generated if j + 1 consecutive I-frames are lost and the previous and next (in presentation order) frames are decoded. The next frame will always be the next I-frame, so this I-frame cannot be lost. The previous frame will always be the last P-frame of the previous GoP, so this P-frame cannot be lost. But any of the P-frames and the I-frame of this GoP cannot be lost too, or the last P-frame will not be decoded. Therefore, $N_{\rm cut}[(j+1) * N + z * (M-1)] = N_G * P_I^{(j+1)} * (1-P_I)^2 (1-P_P)^{N_P}$, where N _G  = F/N is the number of GoPs in the video.

An i * M + z * (M − 1) frames length cut will be generated if the (N _P  + 1 − i)th P-frame of a GoP is lost, and again if previous and next (in presentation order) frames are decoded. Remember that i = 1 ...N _P . The next frame will always be the next I-frame, so this I-frame cannot be lost. The previous frame will be a previous P-frame on the GoP or the I-frame of the GoP, so the previous P-frames and the I-frame cannot be lost. Therefore, $N_{\rm cut}[i * M + z * (M-1)] = N_G * P_P * (1-P_I)^2 (1-P_P)^{N_P-i}$.

A cut of length j * N + i * M + z * (M − 1) frames will be generated if after losing a P-frame the next j I-frames are lost. Then, $N_{\rm cut}[j * N + i * M + z * (M-1)] = N_G * P_I^j * P_P * (1-P_I)^2 (1-P_P)^{N_P-i}$.

To generate a M − 1 frames length cut, the M − 1 frames of a B-frames block have to be lost and its neighbouring frames in presentation order have to be decoded. These neighbouring frames will be the two previous P-frames or the previous I- and P-frame in transmission order. As the P-frames depend on the I-frame and its previous P-frames on the GoP, all the previous P-frames and the I-frame on the GoP cannot be lost. Therefore, for the ith B-frames block the number of cuts of length M − 1 frames is $N_G * P_B^{M-1} * (1-P_I) (1-P_P)^i$, where i = 1 ...N _P .

For an open GoP, the reasoning for B-frames is valid but incomplete, because there are (N _P  + 1) B-frames blocks on the GoP, not only N _P . The frames in this “extra” block depend not only on the I-frame and the N _P P-frames of the GoP, but on the I-frame of the next GoP too. Then, for the (N _P  + 1)th B-frames block of an open GoP the number of cuts of length M − 1 frames is $N_G * P_B^{M-1} * (1-P_I)^2 (1-P_P)^{N_P}$.

In general, the number of cuts of length M − 1 frames is:

$$ N_G * P_B^{M-1} * (1-P_I) \sum\limits_{m=1}^{N_P} (1-P_P)^m + z * N_G * P_B^{M-1} * (1-P_I)^2 (1-P_P)^{N_P} $$

This procedure can be extended to cuts from c = 1 to c = M − 2 frames length. But there is an extra difficulty. Now in each B-frames block there are M − c possible loss combinations that generate a c frames length cut. For example, for the case of c = M − 2 they are two possibilities (see Fig. 7). The first possibility is to lose the first M − 2 B-frames of the block and to not lose the last B-frame. The second possibility is to not lose the first B-frame of the block and to lose the other M − 2 B-frames. In both cases, for the ith B-frames block, the number of cuts of length M − 2 frames is $N_G * (1-P_B) * P_B^{M-2} * (1-P_I) (1-P_P)^i$.

Fig. 7

GoP structure (9, 4) with its two possibilities to generate a two frames cut length

To mathematically express the analytical model of c = 1 ...M − 1 lengths cut, we define σ and δ. σ is the minimum number of B-frames from the same block that should be available to the decoding process (should not be lost) to prevent a cut length larger than c, when the first lost B-frame is the rth in the block. δ is the contribution to the number of cuts of length c frames by the M − c possible loss combinations of a B-frames block that generate a c frames length cut. (20) presents the expression for σ and (21) presents the expression for δ.

$$ \label{eq:sigma2} \sigma = \begin{cases} 0 & \text{ if $r=1$ and $r+c=M$ } \\ 1 & \text{ if $(r=1$ and $r+c<M)$ or $(r>1$ and $r+c=M)$ } \\ 2 & \text{ if $r>1$ and $r+c<M$ } \\ \end{cases} $$

(20)

$$ \label{eq:delta2} \delta = \sum\limits_{r=1}^{M-c} (1-P_B)^{\sigma} $$

(21)

In general, the number of cuts of length c = 1 ...M − 1 frames is:

$$ N_G * \delta * P_B^c * (1-P_I) \sum\limits_{m=1}^{N_P} (1-P_P)^m + z * N_G * P_B^c * (1-P_I)^2 (1-P_P)^{N_P} $$

Equation (22) presents the general expression for N _cut[c]. It depends on the GoP structure of the video and on the loss probabilities of the frames, P _{{I, P, B}}.

$$ \label{eq:Ncut2} N_{\rm cut}[c] = \begin{cases} N_G * \delta * P_B^c (1\!-\!P_I) \: \sum\limits_{m=1}^{N_P} (1\!-\!P_P)^m + \\ + z * N_G * \delta * P_B^c * (1\!-\!P_I)^2 \: (1\!-\!P_P)^{N_P} & {\kern-8pt} \text{ for $c=1 \ldots M-1$ }\\ \\ N_G * P_I^j * P_P * (1\!-\!P_I)^2 \: (1\!-\!P_P)^{N_P-i} & {\kern-8pt} \text{ for $c={\kern-1.5pt} j * N {\kern-1pt} +{\kern-1pt} i * {\kern-1pt} M {\kern-1pt} +{\kern-1pt} z *{\kern-1pt} ({\kern-.5pt} M{\kern-1.5pt} -{\kern-1.5pt} 1{\kern-.5pt} )$}\\ \\ N_G * P_I^j * (1\!-\!P_I)^2 \: (1\!-\!P_P)^{N_P} & {\kern-8pt} \text{ for $c={\kern-1pt} (j+1) * N + z * (M{\kern-.5pt} -{\kern-1.5pt} 1{\kern-.5pt} )$ } \\ \\ 0 & {\kern-8pt} \text{ otherwise } \end{cases} $$

(22)

The total number of cuts T _cut can be computed as:

$$ \label{eq:TotalCuts2} T_{\rm cut} = \sum\limits_{c=1}^{F} N_{\rm cut}[c] $$

(23)

The proportion of cuts of c frames length (P _cut[c]) can be computed dividing the number of cuts of c frames length (N _cut[c]) by the total number of cuts (T _cut). The cut length Probability Mass Function P _cut is the set of all possible values of P _cut[c].

$$ \label{eq:Pcut2} P_{\rm cut}[c] = \frac{ N_{\rm cut}[c] }{ T_{\rm cut} } = \frac{ N_{\rm cut}[c] }{ \sum\limits_{i=1}^{F} N_{\rm cut}[i] } $$

(24)

The average cut length L _cut can be computed from the cut length Probability Mass Function:

$$ \label{eq:CutLength2} L_{\rm cut} = \sum\limits_{c=1}^{F} c * P_{\rm cut}[c] = \frac{ \sum\limits_{c=1}^{F} c * N_{\rm cut}[c] }{ \sum\limits_{c=1}^{F} N_{\rm cut}[c] } = \frac{ \sum\limits_{c=1}^{F} c * N_{\rm cut}[c] }{ T_{\rm cut} } $$

(25)