Spatial parameters for audio coding: MDCT domain analysis and synthesis

Abstract

We use Modified Discrete Cosine Transform (MDCT) to analyze and synthesize spatial parameters. MDCT in itself lacks phase information and energy conservation, which are needed by spatial parameters representation. Completing MDCT with Modified Discrete Sine Transform (MDST) into “MDCT-j*MDST” overcomes this and enables the representation in a form similar to that of DFT. And due to overlap-add in time domain, a MDST spectrum can be built perfectly from MDCT spectra of neighboring frames through matrix-vector multiplication. The matrix is heavily diagonal and keeping only a small number of its sub-diagonals is sufficient for approximation. When using MDCT based core coder in spatial audio coding, like Advanced Audio Coding (AAC), we need no separate transforming for spatial processing, cutting down significantly the computational complexity. Subjective listening tests also show that MDCT domain spatial processing has no quality impairment.

Appendix

1.1 A. MDFT energy conservation

As in (3.a) and (3.b), ${c_0}, \ldots, {c_{N - 1}}\,{\text{and}}\,{s_0}, \ldots, {s_{N - 1}}$ are 2N-dimensional basis vectors for MDCT and MDST respectively. The inner products between them are

$$ \left\{ {\begin{array}{*{20}{c}} {\left\langle {{c_k},{c_l}} \right\rangle = N\delta \left( {k - l} \right),}{k,l = 0, \ldots, N - 1} \\ {\left\langle {{s_k},{s_l}} \right\rangle = N\delta \left( {k - l} \right),}{k,l = 0, \ldots, N - 1} \\ {\left\langle {{c_k},{s_l}} \right\rangle = 0,}{k,l = 0, \ldots, N - 1} \\ \end{array} } \right., $$

(A.1)

where δ(•) is the unit impulse function. They compose an orthogonal basis for 2N-dimensional real vector space. Then for a time signal $x(n),\,n = 0, \ldots, 2N - 1$, and its MDCT spectrum X(k) and MDST spectrum $Y(k),\,k = 0,...,2N - 1$, their energy satisfies

$$ \begin{array}{*{20}{c}} {N\left\langle {x,x} \right\rangle } \\ { = \frac{1}{N}\left\langle {\sum\limits_{k = 0}^{N - 1} {\left( {X(k){c_k} + Y(k){s_k}} \right)}, \sum\limits_{k = 0}^{N - 1} {\left( {X(k){c_k} + Y(k){s_k}} \right)} } \right\rangle } \\ { = \left\langle {X,X} \right\rangle + \left\langle {Y,Y} \right\rangle = \left\langle {X + jY,X + jY} \right\rangle } \\ \end{array} . $$

(A.2)

This verifies that MDFT spectral energy is N times of temporal energy.

1.2 B. MDFT time shift and phase shift

From MDFT definition in (3.c), we have when time signal x(n) has a shift d and satisfies $x\left( {n - 2N} \right) = - x(n)$, its MDFT spectrum as

$$ \begin{array}{*{20}{c}} {\tilde Z(k) = \sum\limits_{n = 0}^{2N - 1} {x\left( {n - d} \right)\exp \left[ { - j\frac{\pi }{N}\left( {n + \frac{1}{2} + \frac{N}{2}} \right)\left( {k + \frac{1}{2}} \right)} \right]} } \\ { = \sum\limits_{n = - d}^{2N - 1 - d} {x(n)\exp \left[ { - j\frac{\pi }{N}\left( {n + d + \frac{1}{2} + \frac{N}{2}} \right)\left( {k + \frac{1}{2}} \right)} \right]} } \\ { = \sum\limits_{n = 0}^{2N - 1 - d} {x(n)\exp \left[ { - j\frac{\pi }{N}\left( {n + d + \frac{1}{2} + \frac{N}{2}} \right)\left( {k + \frac{1}{2}} \right)} \right]} } \\ { - \sum\limits_{n = 2N - d}^{2N - 1} {x\left( {n - 2N} \right)\exp \left[ { - j\frac{\pi }{N}\left( {n + d + \frac{1}{2} + \frac{N}{2}} \right)\left( {k + \frac{1}{2}} \right)} \right]} } \\ { = Z(k)\exp \left[ { - j\frac{\pi }{N}d\left( {k + \frac{1}{2}} \right)} \right]} \\ \end{array}, $$

(A.3)

where Z(k) is MDFT spectrum of x(n) without shift. The condition $x\left( {n - 2N} \right) = - x(n)$ parallels DFT’s requirement of periodicity but with a negative sign. For real signals and d<<2N, (A.4) is an approximation.

1.3 C. Windowed MDFT

Note X(k) and Y(k) are sine-windowed MDCT spectrum and cosine-windowed MDST spectrum respectively. Then we have

$$ \begin{array}{*{20}{c}} {{Z_{+} }(k) = Y(k) + X(k)} \\ { = \sum\limits_{n = 0}^{2N - 1} {x(n)\cos \left[ {\frac{\pi }{2N}\left( {n + \frac{1}{2}} \right)} \right]\sin \left[ {\frac{\pi }{N}\left( {n + \frac{1}{2} + \frac{N}{2}} \right)\left( {k + \frac{1}{2}} \right)} \right]} } \\ { + \sum\limits_{n = 0}^{2N - 1} {x(n)\sin \left[ {\frac{\pi }{2N}\left( {n + \frac{1}{2}} \right)} \right]\cos \left[ {\frac{\pi }{N}\left( {n + \frac{1}{2} + \frac{N}{2}} \right)\left( {k + \frac{1}{2}} \right)} \right]} } \\ { = - \sum\limits_{n = 0}^{2N - 1} {x(n)\cos \left[ {\frac{\pi }{N}n\left( {k + 1} \right) + \frac{\pi }{N}\left( {k + 1} \right)\left( {\frac{1}{2} + \frac{N}{2}} \right) + \frac{\pi }{4}} \right]} } \\ \end{array}, $$

(A.4)

and

$$ \begin{array}{*{20}{c}} {{Z_{-} }(k) = Y(k) - X(k)} \\ { = \sum\limits_{n = 0}^{2N - 1} {x(n)\cos \left[ {\frac{\pi }{2N}\left( {n + \frac{1}{2}} \right)} \right]\sin \left[ {\frac{\pi }{N}\left( {n + \frac{1}{2} + \frac{N}{2}} \right)\left( {k + \frac{1}{2}} \right)} \right]} } \\ { - \sum\limits_{n = 0}^{2N - 1} {x(n)\sin \left[ {\frac{\pi }{2N}\left( {n + \frac{1}{2}} \right)} \right]\cos \left[ {\frac{\pi }{N}\left( {n + \frac{1}{2} + \frac{N}{2}} \right)\left( {k + \frac{1}{2}} \right)} \right]} } \\ { = \sum\limits_{n = 0}^{2N - 1} {x(n)\sin \left[ {\frac{\pi }{N}nk + \frac{\pi }{N}k\left( {\frac{1}{2} + \frac{N}{2}} \right) + \frac{\pi }{4}} \right]} } \\ \end{array} . $$

(A.5)

Take (A.4) and (A.5) as real part and imaginary part respectively,

$$ \begin{array}{*{20}{c}} { - {Z_{+} }\left( {k - 1} \right) - j{Z_{-} }(k)} \\ { = \sum\limits_{n = 0}^{2N - 1} {x(n)\cos \left[ {\frac{\pi }{N}nk + \frac{\pi }{N}k\left( {\frac{1}{2} + \frac{N}{2}} \right) + \frac{\pi }{4}} \right]} } \\ { - j\sum\limits_{n = 0}^{2N - 1} {x(n)\sin \left[ {\frac{\pi }{N}nk + \frac{\pi }{N}k\left( {\frac{1}{2} + \frac{N}{2}} \right) + \frac{\pi }{4}} \right]} } \\ { = \exp \left\{ { - j\left[ {\frac{\pi }{N}k\left( {\frac{1}{2} + \frac{N}{2}} \right) + \frac{\pi }{4}} \right]} \right\}\sum\limits_{n = 0}^{2N - 1} {x(n)\exp \left[ { - j\frac{\pi }{N}nk} \right]} } \\ \end{array}, $$

(A.6)

which is 2N-point DFT with a phase shift. Moreover with ${Z_{+} }\left( { - 1} \right) = - {Z_{-} }(0)\,{\text{and}}\,{Z_{-} }(N) = {Z_{+} }\left( {N - 1} \right)$, (A.6) leads to (5.a).

1.4 D. Properties of MDCT and MDST transform matrices

From (6), we can see each column vector of C ₀ and S ₁ are odd-symmetric, and each column vector of C ₁ and S ₀ are even-symmetric. With the help of anti-diagonal matrix J having only 1 on its anti-diagonal, the symmetries are equivalent to ${\mathbf{J}}{{\mathbf{C}}_0} = - {{\mathbf{C}}_0},{\mathbf{J}}{{\mathbf{S}}_1} = - {{\mathbf{S}}_1}\,{\text{and}}\,{\mathbf{J}}{{\mathbf{C}}_1} = {{\mathbf{C}}_1},{\mathbf{J}}{{\mathbf{S}}_0} = {{\mathbf{S}}_0}$ respectively. From this and ${{\mathbf{J}}^{\text{T}}}{\mathbf{J}} = {\mathbf{JJ}} = {\mathbf{I}}$, we have

$$ {\mathbf{S}}_0^{\text{T}}{{\mathbf{C}}_0} = {\mathbf{S}}_0^{\text{T}}{{\mathbf{J}}^{\text{T}}}{\mathbf{J}}{{\mathbf{C}}_0} = {\left( {{\mathbf{J}}{{\mathbf{S}}_0}} \right)^{\text{T}}}\left( {{\mathbf{J}}{{\mathbf{C}}_0}} \right) = - {\mathbf{S}}_0^{\text{T}}{{\mathbf{C}}_0}, $$

(A.7)

which implies ${\mathbf{S}}_0^{\text{T}}{{\mathbf{C}}_0} = {\mathbf{0}}$. And for the same reason, ${\mathbf{S}}_1^{\text{T}}{{\mathbf{C}}_1} = {\mathbf{0}}$. For the windowed case, from the second equation of (14) W ₁=JW ₀ J and that W ₀ and W ₁ are diagonal matrices then W ₀ W ₁=W ₁ W ₀, we have

$$ \begin{array}{*{20}{c}} {{\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_1}{{\mathbf{W}}_0}{{\mathbf{C}}_0} = {\mathbf{S}}_0^{\text{T}}{{\mathbf{J}}^{\text{T}}}{\mathbf{J}}{{\mathbf{W}}_1}{\mathbf{JJ}}{{\mathbf{W}}_0}{\mathbf{JJ}}{{\mathbf{C}}_0}} \\ { = {{\left( {{\mathbf{J}}{{\mathbf{S}}_0}} \right)}^{\text{T}}}\left( {{\mathbf{J}}{{\mathbf{W}}_1}{\mathbf{J}}} \right)\left( {{\mathbf{J}}{{\mathbf{W}}_0}{\mathbf{J}}} \right)\left( {{\mathbf{J}}{{\mathbf{C}}_0}} \right)} \\ { = - {\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_0}{{\mathbf{W}}_1}{{\mathbf{C}}_0}} \\ { = - {\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_1}{{\mathbf{W}}_0}{{\mathbf{C}}_0}} \\ \end{array}, $$

(A.8)

which implies ${\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_1}{{\mathbf{W}}_0}{{\mathbf{C}}_0} = {\mathbf{0}}$. And for the same reason, ${\mathbf{S}}_1^{\text{T}}{{\mathbf{W}}_0}{{\mathbf{W}}_1}{{\mathbf{C}}_1} = {\mathbf{0}}$. Also by similar procedure as (A.8), we have ${\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_1}{{\mathbf{W}}_1}{{\mathbf{C}}_1} = {\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_0}{{\mathbf{W}}_0}{{\mathbf{C}}_1}$. From this and with the help of the first equation of (14) ${{\mathbf{W}}_0}{{\mathbf{W}}_0} + {{\mathbf{W}}_1}{{\mathbf{W}}_1} = {\mathbf{I}}$, we can see

$$ \begin{array}{*{20}{c}} {{\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_1}{{\mathbf{W}}_1}{{\mathbf{C}}_1} = \frac{1}{2}\left( {{\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_0}{{\mathbf{W}}_0}{{\mathbf{C}}_1} + {\mathbf{S}}_0^{\text{T}}{{\mathbf{W}}_1}{{\mathbf{W}}_1}{{\mathbf{C}}_1}} \right)} \\ { = \frac{1}{2}{\mathbf{S}}_0^{\text{T}}\left( {{{\mathbf{W}}_0}{{\mathbf{W}}_0} + {{\mathbf{W}}_1}{{\mathbf{W}}_1}} \right){{\mathbf{C}}_1}} \\ { = \frac{1}{2}{\mathbf{S}}_0^{\text{T}}{{\mathbf{C}}_1}} \\ \end{array} . $$

(A.9)

And for the same reason, ${\mathbf{S}}_1^{\text{T}}{{\mathbf{W}}_0}{{\mathbf{W}}_0}{{\mathbf{C}}_0} = {\mathbf{S}}_1^{\text{T}}{{\mathbf{C}}_0}/2$.

1.5 E. Properties of the conversion matrix T

As in (7.b), P is a matrix having only $+ 1, - 1, + 1, - 1, \ldots,$ on its diagonal, implying PP ^T=I. And with ${{\mathbf{S}}_0} = - {{\mathbf{C}}_1}{\mathbf{P}},{{\mathbf{S}}_1} = {{\mathbf{C}}_0}{\mathbf{P}}$ in (7.b), we have ${{\mathbf{S}}_1}{\mathbf{S}}_1^{\text{T}} = {{\mathbf{C}}_0}{\mathbf{C}}_0^{\text{T}},\,{{\mathbf{S}}_0}{\mathbf{S}}_0^{\text{T}} = {{\mathbf{C}}_1}{\mathbf{C}}_1^{\text{T}},\,{{\mathbf{S}}_0}{\mathbf{S}}_1^{\text{T}} = - {{\mathbf{C}}_1}{\mathbf{C}}_0^{\text{T}},\,{{\mathbf{S}}_1}{\mathbf{S}}_0^{\text{T}} = - {{\mathbf{C}}_0}{\mathbf{C}}_1^{\text{T}}$. With the help of ${\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1} = N{\mathbf{I}}\,{\text{and}}\,{{\mathbf{C}}_1}{\mathbf{C}}_0^T = {{\mathbf{C}}_0}{\mathbf{C}}_1^T = {\mathbf{0}}$ in (7.a), the conversion matrix defined in (10.b) is orthogonal, or

$$ \begin{array}{*{20}{c}} {{{\mathbf{T}}^{\text{T}}}{\mathbf{T}} = \frac{1}{N^2}{{\left( {{\mathbf{S}}_1^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{S}}_0^{\text{T}}{{\mathbf{C}}_1}} \right)}^{\text{T}}}\left( {{\mathbf{S}}_1^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{S}}_0^{\text{T}}{{\mathbf{C}}_1}} \right)} \\ { = \frac{1}{N^2}\left( {{\mathbf{C}}_0^{\text{T}}{{\mathbf{S}}_1}{\mathbf{S}}_1^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{C}}_1^{\text{T}}{{\mathbf{S}}_0}{\mathbf{S}}_0^{\text{T}}{{\mathbf{C}}_1} + {\mathbf{C}}_1^{\text{T}}{{\mathbf{S}}_0}{\mathbf{S}}_1^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{C}}_0^{\text{T}}{{\mathbf{S}}_1}{\mathbf{S}}_0^{\text{T}}{{\mathbf{C}}_1}} \right)} \\ { = \frac{1}{N^2}\left( {{\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0}{\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1}{\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1} - {\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1}{\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0} - {\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0}{\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1}} \right)} \\ { = \frac{1}{N^2}\left( {{\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0}{\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1}{\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1} + {\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1}{\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0}{\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1}} \right)} \\ { = \frac{1}{N^2}\left( {{\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1}} \right)\left( {{\mathbf{C}}_0^{\text{T}}{{\mathbf{C}}_0} + {\mathbf{C}}_1^{\text{T}}{{\mathbf{C}}_1}} \right)} \\ { = {\mathbf{I}}} \\ \end{array} . $$

(A.10)