Matrix iterative solutions to the least squares problem of BXAT = F with some linear constraints
Introduction
We consider numerical methods for solving the matrix-form least squares problem with linear constraintswhere matrices , , and are given, denotes a set of constrained matrices by preindicated linear constraints such as matrix structures. In theory, using Kronecker production ⊗, the constrained matrix least squares problem (1.1) can be equivalently rewritten as a constrained LS in vector-form [3]where vec(X) denotes the long vector consisting of all columns of X. Clearly, this equivalent transformation results in a coefficient matrix in large scale and increases computational complexity and storage requirement. It hence cannot be a practicable method for solving the constrained matrix LS problem if the system scale is large.
There are some valuable efforts on formulating solutions to the matrix-form least squares problem with or without linear constraints [1], [2], [7], [8], [11], [13], [14], [15]. Inevitably, Moore–Penrose generalized inverses and some complicated matrix decompositions such as canonical correlation decomposition (CCD) [5] and general singular value decomposition (GSVD) [10] are involved. It is also required to solve a matrix equation that may be ill-conditioned. Because of the obvious difficulties in numerical instability and computational complexity, those constructional solutions narrow down their applications. Indeed, it is impractical to find a solution by those formulas if the matrix size is large.
In this paper, we focus on matrix iteration methods for solving the matrix LS problem (1.1) with linear constraints. In [16], a matrix iteration method was given for solving (1.1) with the symmetry constraint XT = X. It is a matrix-form CGLS method for the (unconstrained) matrix LS problem minX∥BXAT − F∥F. The symmetry condition is automatically satisfied by the iterative matrices if the initial guess X0 is set to be symmetric. In the classical CGLS [12], the Conjugate Gradient (CG) method applied on the normal equation of a linear system, it is only required to compute a residual matrix of the normal equation and update the iterative solution and a conjugate gradient matrix linearly in each iteration. A matrix-form CGLS can be easily derived from the classical CGLS applied on the vector-representation of the matrix LS using Kronecker-product. However, as known, the condition number is square when normal equation is involved. This may lead to numerical instability.
We will use the algorithm LSQR proposed by Paige and Saunders [9] as the frame method for deriving a matrix iteration method for the constrained LS problem (1.1) because of favorable numerical properties of LSQR. The basic idea is that for a given linear constraint, we characterize the constrained matrix by vec(X) = CSx in terms of its independent parameter vector x of X, where CS is a full column-rank matrix that is called a constraint matrix. This relation determines a linear mapping from the constraint space to its independent parameter space. By the Kronecker product of A and B, the constrained matrix LS problem can be equivalently transformed to an unconstrained LS problem in vector form with coefficient matrix M = (A ⊗ B)CS, and hence it can be solved by a Krylov subspace method theoretically.
LSQR involves two basic operations Mx and MTy. To transform the vector-form iteration into a matrix-form LSQR for solving the matrix LS problem with constraints, and to get rid of Kronecker production, it is required to represent Mx in an m2 × m1 matrix in terms of matrix–matrix products, and transform MTy back to an n × n matrix in the constraint set (the inverse mapping of MTy) by products of matrices, too. For , the matrix representation of Mx is given by BXAT. However it is not trivial for representing in terms of matrix–matrix products. We will give detailed analysis of the constraint matrices corresponding to variant linear constraints such as symmetry, skew-symmetry, and symmetric/skew-symmetric P-commuting. Special structures of the constraint matrices considered in this paper will be exploited and used to represent in matrix–matrix products without Kronecker product.
The rest of paper is organized as follows. In Section 2, we characterize the symmetry or skew-symmetry constraint matrices which map the constrained matrix spaces to their parameter vector spaces. Constraint matrices for symmetric or skew-symmetric P-commuting will be discussed in Section 3. In Section 4, we shortly review the related algorithms LSQR and CGLS. Our matrix iterative algorithms for the constrained matrix LS problem are proposed in Section 5, based on the classical LSQR. We give detailed discussions to represent using the properties shown in Sections 2 Symmetry and skew-symmetry constraint matrices, 3 Constraint matrices for symmetry or skew-symmetry for the symmetric, skew-symmetric, and symmetric/skew-symmetric P-commuting constraints, respectively. Numerical examples are given in Section 6 to display the efficiency of the algorithms.
Notation. For , we denote by the long vector expanded by columns of X. The sub-vector consisting of from αth component to βth component of xi is denoted by xα:β,i. X+ is the Moore–Penrose generalized inverse of X. ∥·∥F is the Frobenius norm of matrix, while ∥ · ∥2 is the 2-norm of vector or matrix. We also denote by sym(X) and ssym(X) the symmetric part and the skew-symmetric part of X, respectively.
Section snippets
Symmetry and skew-symmetry constraint matrices
The symmetry or skew-symmetry are two commonly imposed linear constraints upon the matrix least squares problem [2], [8]. A symmetric matrix is uniquely and linearly determined by the elements of its lower triangular part, while a skew-symmetric matrix is constitutive of the elements of its strictly lower triangular part. For a square matrix , we denote by vecL(X) and vecR(X) the column vectors consisting of the components in the lower triangular part and the strictly lower
Constraint matrices for symmetry or skew-symmetry P-commuting constraints
Let P be a given square matrix. Consider the set of matrices that are commutate with P,For the (real) R-symmetric Procrustes problem, P is a symmetric and orthogonal matrix [14]. It is known that for a symmetric P, based on the eigenvalue decompositionof P, where p is different eigenvalues λi of multiples ki, i = 1, … , p, a matrix X is commuting to P if and only ifwhere . Clearly, Xi are the independent parameter matrices if no other
The CGLS and LSQR algorithms
We briefly discuss the two mathematically equivalent algorithms CGLS proposed by Stiefel [12] and LSQR by Paige and Saunders [9] for solving the following least squares problem:with given and . The CGLS method gives a geometric understanding, meanwhile LSQR presents elegant algorithmic techniques. CGLS algorithm is the Conjugate Gradient (CG) method [6] applied on the normal equationStarting at an initial guess x0, CGLS finds an optimal solution xk in the affine
Matrix iterations of LSQR for constrained matrix LS problems
The constrained matrix LS problem (1.1) can be equivalently transformed to an LS problem in vector form without constraints. Let be the given constraint set and CS is the constraint matrix corresponding to the linear mapping from to its parameter space such that for Obviously, CS is of full column rank. Sincewhere ⊗ denote the Kronecker production, then we have vec(BXAT) = (A ⊗ B)CSx and the constrained problem (1.1) is equivalent to
Numerical example
In this section, we present some numerical examples to illustrate the efficiency of the matrix iteration methods based upon LSQR. The test matrices include the data shown in [16] and matrices that are randomly constructed with chosen singular values. For example,with orthogonal matrices U and V constructed as follows (in MATLAB notation):The singular values {σi} will be designed so that the resulting coefficient matrices A and B
References (16)
- et al.
Perturbation analysis of the canonical correlation of matrix pairs
Linear Algebra Appl.
(1994) Computing a nearest symmetric positive semidefinite matrix
Linear Algebra Appl.
(1988)Hermitian, hermitian R-symmetric, and hermitian R-skew symmetric Procrustes problems
Linear Algebra Appl.
(2004)- et al.
On solutions of matrix equation AXB + CYD = F
Linear Algebra Appl.
(1998) - et al.
A constrained Procrustes problem
SIAM J. Matrix Anal. Appl.
(1997) - et al.
Least squares solutions of BXAT = T over symmetric, skew- symmetric, and positive semidefinite X∗
SIAM J. Matrix Anal. Appl.
(2003) - et al.
Large least squares problems involving Kronecker products
SIAM J. Matrix Anal. Appl.
(1994) - et al.
Calculating the singular values and pseudoinverse of a matrix
SIAM J. Numer. Anal.
(1965)
Cited by (20)
An iteration method to solve multiple constrained least squares problems
2017, Journal of Computational and Applied MathematicsIterative methods for solving consistent or inconsistent matrix inequality AXB≥C with linear constraints
2015, Applied Mathematical ModellingCitation Excerpt :There are many valuable iterative methods on formulating the solutions and least squares solutions of various matrix equations with or without linear constraints [4–9]. Moreover, many numerical experiments show that the matrix-form LSQR iterative method [10–13] is more efficient than others due to its favorable numerical properties, such as the low storage and numerical stability. To our knowledge, however, there are few efficient numerical methods for solving the matrix inequality with constraints thus far.
Solving linearly constrained matrix least squares problem by LSQR
2014, Applied Mathematics and ComputationNew symmetry preserving method for optimal correction of damping and stiffness matrices using measured modes
2010, Journal of Computational and Applied MathematicsLeast squares solution with the minimum-norm to general matrix equations via iteration
2010, Applied Mathematics and ComputationWeighted least squares solutions to general coupled Sylvester matrix equations
2009, Journal of Computational and Applied Mathematics
- 1
The work of this author was supported in part by NSFC (project 60372033).