A Jacobi-like joint diagonalization method by one-dimensional optimization
Introduction
The issue of approximate joint diagonalization (JD) of a set of target matrices has been researched extensively in the past few decades, which has used to implement many blind source separation (BSS) algorithms as in [1], [2], [3], [4], [5], [6], [7], independent component analysis (ICA) as in [8], [9], and tensor decomposition as in [10], etc. The JD methods can be classified into two categories: orthogonal joint diagonalization (OJD) and non-orthogonal joint diagonalization (NOJD). In the first category, these algorithms like JADE [11], SOBI [1], require the mixing matrix to be orthonormal. In these algorithms, the pre-whitening step is exploited to transform the unknown mixture matrix into some orthonormal (or unitary in the complex case) matrices. In the second category, the NOJD techniques, dealing with the JD problem without pre-whitening process, have been considered in the literature including JDi algorithm [2], s-BIA algorithm [3], CJDi algorithm [6], FFDIAG algorithm [12], ACDC algorithm [13] and U-WEDGE [14], etc.
Especially, the noise sensitivity of JD methods researched by Bijan Afsari (see, e.g., [15], [16], [17]) shows that the NOJD methods are more difficult than OJD both in the sense of algorithmic deduction and the solution behavior. There are two main reasons for this. On one hand, by whitening the data, the mutual information is reduced, so that the whitened data is closer to be independent. On the other hand, whitening the data reduces the dynamic range of the norm of target matrices and enables better convergence of the OJD methods.
There are mainly three contributions in this paper. Firstly, we propose a simple and efficient Jacobi-like algorithm for real-valued orthogonal joint diagonalization. The diagonalizer is constructed as the product of a set of modified Givens rotations that dependent on a single parameter , instead of a Jacobi angle [1], [2], [11], [18], [19], [20]. By this formulation, the local cost function with respect to each index pair can be expressed as a simple function of s, whose maximum can be obtained in closed-form by one-dimensional global analytical search. Secondly, the real-valued algorithm can be slightly modified and generalized to complex case since the target matrices can be transformed into real symmetric ones in the same way as [6]. By taking advantage of the special structure of the transformed target matrices, the computational cost can be significantly reduced. Thirdly, the proposed algorithm can be directly handle the complex-valued joint diagonalization problem by a bi-Givens strategy, since the elements of the modified rotation matrices are real or pure imaginary, the computational cost is not increased comparing with the classical SOBI algorithm. The simulation results confirm that the proposed algorithms may attain comparatively good performance, and show that the solution obtained by the proposed complex OJD algorithm is essentially equal to that given by SOBI.
The rest of this paper is organized as follows. In Section 2, the problem formulation is stated. The proposed real-valued OJD algorithm is described in Section 3. Section 4 is dedicated to complex implementation of the real-valued algorithm. The simulation results in comparison with some elegant methods are illustrated in Section 5. Section 6 is devoted to conclusions.
Section snippets
Problem formulation
Mathematically, the OJD problem considered here can be stated as follows: given a set of matrices , sharing the same structure defined bywhere are diagonal matrices, A is an unknown orthonormal (or unitary in complex case) mixing matrix, the superscript denotes the conjugate transpose in the complex-valued case and transpose in the real-valued case. The target matrices can be taken as the covariance matrices estimated on different time delays,
Proposed real-valued OJD algorithm
In this section, we propose an iterative algorithm with exact expression of optimal rotation parameter at each step. It is observed that the square of Frobenius norm of a matrix is invariant under an orthonormal (or unitary) similarity transformation. This invariance allows us to equivalently convert the JD problem of minimizing the sum of the square of the off-diagonal elements into a problem of maximizing the sum of square of the diagonal elements, which follows:
Complex implementation
In this section, we present two strategies to seek the joint diagonalizer of a set of complex-valued matrices by extending the real-valued algorithm proposed in Section 3 to the complex-valued space. The first algorithm provided in Section 4.1 adopts the basic idea of [6], by which the real symmetric target matrices are obtained from the complex-valued target matrices. In Section 4.2, we describe a direct complex-valued algorithm based on bi-Givens rotation (BGR) of the target matrices.
Simulation results
This section is devoted to the performance evaluation of the proposed algorithms, in several matrix perturbed cases. We illustrate the performance of the proposed algorithms, in comparison with three other Jacobi-like algorithms namely J-Di [2] in real-valued case, and SOBI [1] and CJDi [6] in complex-valued case. We also consider s-BIA [3] and U-WEDGE [14] in both real-valued and complex-valued cases. The computational costs of these algorithms are evaluated in terms of the number of real
Conclusions
In this paper, the orthogonal joint diagonalization based on Givens rotation has been improved by one-dimensional global analytical search. The desired diagonalizing matrix has been constructed by successive multiplications of Jacobi transformation matrices that depend on a single parameter. The closed-form solution of the single parameter has been obtained by certain algebraic derivation procedure in real-valued case. This algorithm is indirectly applicable to complex case if the complex
Acknowledgments
The authors would like to thank very much the Associate Editor Professor S. Chen and the anonymous reviewers very much for their valuable comments and suggestions that have significantly improved the manuscript.
This work was supported in part by the National Natural Science Foundation of China under Grant 61271293, 61373177 and by the Scientific Research Plan of Education Department of Shaanxi Province under Grant 11JK0903.
References (24)
- et al.
Joint blind source separation by generalized joint diagonalization of cumulant matrices
Signal Process.
(2011) - et al.
A recursive solution to nonunitary joint diagonalization
Signal Process.
(2013) - et al.
Non-orthogonal joint diagonalization algorithm based on hybrid trust region method and its application to blind source separation
Neurocomputing
(2014) - et al.
Semi-nonnegative joint diagonalization by congruence and semi-nonnegative ICA
Signal Process.
(2014) - et al.
A blind source separation technique using second-order statistics
IEEE Trans. Signal Process.
(1997) Nonorthogonal joint diagonalization by combining givens and hyperbolic rotations
IEEE Trans. Signal Process.
(2009)- et al.
Bi-iterative algorithm for extracting independent components from array signals
IEEE Trans. Signal Process.
(2011) - et al.
A new algorithm for complex non-orthogonal joint diagonalization based on shear and givens rotations
IEEE Trans. Signal Process.
(2014) - et al.
Quaternion ICA from second-order statistics
IEEE Trans. Signal Process.
(2011) A link between the canonical decomposition in multilinear algebra and simultaneous matrix diagonalization
SIAM J. Matrix Anal. Appl.
(2006)
Blind beamforming for non-Gaussian signals
IEE Proc. F: Radar Signal Process.
A fast algorithm for joint diagonalization with non-orthogonal transformations and its application to blind source separation
J. Mach. Learn. Res.
Cited by (2)
Fast Hilbert-Wavelet Simulation of Nonstationary Wind Field Using Noniterative Simultaneous Matrix Diagonalization
2021, Journal of Engineering MechanicsA two-stage dual-matrix method of blind signal separation
2017, Signal, Image and Video Processing