Matrix iterative solutions to the least squares problem of BXAT = F with some linear constraints

https://doi.org/10.1016/j.amc.2006.06.097Get rights and content

Abstract

Matrix iterative algorithms are proposed for solving the matrix least square problem of BXAT = F with variant linear constraints on solutions such as symmetry, skew-symmetry, and symmetry/skew-symmetry commuting with a given symmetric matrix P. We characterize the linear mapping from the constrained solution sets to their (independent) parameter spaces, and use these properties to deduce the matrix iterations, based on the classical algorithm LSQR for solving (unconstrained) LS problem. Numerical results are reported that show the efficiency of the proposed methods.

Introduction

We consider numerical methods for solving the matrix-form least squares problem with linear constraintsminXSBXAT-FF,where matrices ARm1×n, BRm2×n, and FRm2×m1 are given, S denotes a set of constrained matrices by preindicated linear constraints such as matrix structures. In theory, using Kronecker production ⊗, the constrained matrix least squares problem (1.1) can be equivalently rewritten as a constrained LS in vector-form [3]minXS(AB)vec(X)-vec(F)2,where vec(X) denotes the long vector consisting of all columns of X. Clearly, this equivalent transformation results in a coefficient matrix in large scale and increases computational complexity and storage requirement. It hence cannot be a practicable method for solving the constrained matrix LS problem if the system scale is large.

There are some valuable efforts on formulating solutions to the matrix-form least squares problem with or without linear constraints [1], [2], [7], [8], [11], [13], [14], [15]. Inevitably, Moore–Penrose generalized inverses and some complicated matrix decompositions such as canonical correlation decomposition (CCD) [5] and general singular value decomposition (GSVD) [10] are involved. It is also required to solve a matrix equation that may be ill-conditioned. Because of the obvious difficulties in numerical instability and computational complexity, those constructional solutions narrow down their applications. Indeed, it is impractical to find a solution by those formulas if the matrix size is large.

In this paper, we focus on matrix iteration methods for solving the matrix LS problem (1.1) with linear constraints. In [16], a matrix iteration method was given for solving (1.1) with the symmetry constraint XT = X. It is a matrix-form CGLS method for the (unconstrained) matrix LS problem minXBXAT  FF. The symmetry condition is automatically satisfied by the iterative matrices if the initial guess X0 is set to be symmetric. In the classical CGLS [12], the Conjugate Gradient (CG) method applied on the normal equation of a linear system, it is only required to compute a residual matrix of the normal equation and update the iterative solution and a conjugate gradient matrix linearly in each iteration. A matrix-form CGLS can be easily derived from the classical CGLS applied on the vector-representation of the matrix LS using Kronecker-product. However, as known, the condition number is square when normal equation is involved. This may lead to numerical instability.

We will use the algorithm LSQR proposed by Paige and Saunders [9] as the frame method for deriving a matrix iteration method for the constrained LS problem (1.1) because of favorable numerical properties of LSQR. The basic idea is that for a given linear constraint, we characterize the constrained matrix XS by vec(X) = CSx in terms of its independent parameter vector x of X, where CS is a full column-rank matrix that is called a constraint matrix. This relation determines a linear mapping L:Xx=(CS)+vec(X) from the constraint space S to its independent parameter space. By the Kronecker product of A and B, the constrained matrix LS problem can be equivalently transformed to an unconstrained LS problem in vector form with coefficient matrix M = (A  B)CS, and hence it can be solved by a Krylov subspace method theoretically.

LSQR involves two basic operations Mx and MTy. To transform the vector-form iteration into a matrix-form LSQR for solving the matrix LS problem with constraints, and to get rid of Kronecker production, it is required to represent Mx in an m2 × m1 matrix in terms of matrix–matrix products, and transform MTy back to an n × n matrix in the constraint set S (the inverse mapping L-1(MTy) of MTy) by products of matrices, too. For x=L(X), the matrix representation of Mx is given by BXAT. However it is not trivial for representing L-1(MTy) in terms of matrix–matrix products. We will give detailed analysis of the constraint matrices corresponding to variant linear constraints such as symmetry, skew-symmetry, and symmetric/skew-symmetric P-commuting. Special structures of the constraint matrices considered in this paper will be exploited and used to represent L-1(MTy)S in matrix–matrix products without Kronecker product.

The rest of paper is organized as follows. In Section 2, we characterize the symmetry or skew-symmetry constraint matrices which map the constrained matrix spaces to their parameter vector spaces. Constraint matrices for symmetric or skew-symmetric P-commuting will be discussed in Section 3. In Section 4, we shortly review the related algorithms LSQR and CGLS. Our matrix iterative algorithms for the constrained matrix LS problem are proposed in Section 5, based on the classical LSQR. We give detailed discussions to represent L-1(MTy)S using the properties shown in Sections 2 Symmetry and skew-symmetry constraint matrices, 3 Constraint matrices for symmetry or skew-symmetry for the symmetric, skew-symmetric, and symmetric/skew-symmetric P-commuting constraints, respectively. Numerical examples are given in Section 6 to display the efficiency of the algorithms.

Notation. For X=[x1,,xn]Rm×n, we denote by vec(X)=[x1T,,xnT]T the long vector expanded by columns of X. The sub-vector consisting of from αth component to βth component of xi is denoted by xα:β,i. X+ is the Moore–Penrose generalized inverse of X. ∥·∥F is the Frobenius norm of matrix, while ∥ · 2 is the 2-norm of vector or matrix. We also denote by sym(X) and ssym(X) the symmetric part sym(X)=12(X+XT) and the skew-symmetric part 12(X-XT) of X, respectively.

Section snippets

Symmetry and skew-symmetry constraint matrices

The symmetry or skew-symmetry are two commonly imposed linear constraints upon the matrix least squares problem [2], [8]. A symmetric matrix is uniquely and linearly determined by the elements of its lower triangular part, while a skew-symmetric matrix is constitutive of the elements of its strictly lower triangular part. For a square matrix X=(xij)Rn×n, we denote by vecL(X) and vecR(X) the column vectors consisting of the components in the lower triangular part and the strictly lower

Constraint matrices for symmetry or skew-symmetry P-commuting constraints

Let P be a given square matrix. Consider the set of matrices that are commutate with P,S={XPX=XP}.For the (real) R-symmetric Procrustes problem, P is a symmetric and orthogonal matrix [14]. It is known that for a symmetric P, based on the eigenvalue decompositionP=Qdiag(λ1Ik1,,λpIkp)QTof P, where p is different eigenvalues λi of multiples ki, i = 1,  , p, a matrix X is commuting to P if and only ifX=Qdiag(X1,,Xp)QT,where XiRki×ki. Clearly, Xi are the independent parameter matrices if no other

The CGLS and LSQR algorithms

We briefly discuss the two mathematically equivalent algorithms CGLS proposed by Stiefel [12] and LSQR by Paige and Saunders [9] for solving the following least squares problem:minxMx-f2,with given MRm×n and fRm. The CGLS method gives a geometric understanding, meanwhile LSQR presents elegant algorithmic techniques. CGLS algorithm is the Conjugate Gradient (CG) method [6] applied on the normal equationMTMx=MTf.Starting at an initial guess x0, CGLS finds an optimal solution xk in the affine

Matrix iterations of LSQR for constrained matrix LS problems

The constrained matrix LS problem (1.1) can be equivalently transformed to an LS problem in vector form without constraints. Let S be the given constraint set and CS is the constraint matrix corresponding to the linear mapping L from S to its parameter space such that for XSvec(X)=CSx,x=L(X).Obviously, CS is of full column rank. Sincevec(BXAT)=(AB)vec(X),where ⊗ denote the Kronecker production, then we have vec(BXAT) = (A  B)CSx and the constrained problem (1.1) is equivalent tominx(AB)CSx-vec(

Numerical example

In this section, we present some numerical examples to illustrate the efficiency of the matrix iteration methods based upon LSQR. The test matrices include the data shown in [16] and matrices that are randomly constructed with chosen singular values. For example,A=Udiag(σ1,,σn)VTwith orthogonal matrices U and V constructed as follows (in MATLAB notation):[U,temp]=qr(1-2rand(n));[V,temp]=qr(1-2rand(n)).The singular values {σi} will be designed so that the resulting coefficient matrices A and B

References (16)

There are more references available in the full text version of this article.

Cited by (20)

  • An iteration method to solve multiple constrained least squares problems

    2017, Journal of Computational and Applied Mathematics
  • Iterative methods for solving consistent or inconsistent matrix inequality AXB≥C with linear constraints

    2015, Applied Mathematical Modelling
    Citation Excerpt :

    There are many valuable iterative methods on formulating the solutions and least squares solutions of various matrix equations with or without linear constraints [4–9]. Moreover, many numerical experiments show that the matrix-form LSQR iterative method [10–13] is more efficient than others due to its favorable numerical properties, such as the low storage and numerical stability. To our knowledge, however, there are few efficient numerical methods for solving the matrix inequality with constraints thus far.

  • Weighted least squares solutions to general coupled Sylvester matrix equations

    2009, Journal of Computational and Applied Mathematics
View all citing articles on Scopus
1

The work of this author was supported in part by NSFC (project 60372033).

View full text