Brief paperConsistency of subspace methods for signals with almost-periodic components☆
Introduction
This paper deals with subspace identification of stationary processes with oscillatory components. At a first sight this problem may look like a minor generalization of a standard identification problem which has been exhaustively treated in the literature since the early 1990s. In reality, on one hand the problem encompasses harmonic retrieval; that is, estimation of the harmonic components of a stationary signal in additive noise, a problem of paramount importance in signal processing which, in the multichannel case, cannot be approached by the standard methods like Pisarenko, MUSIC, ESPRIT etc. It seems fair to say that the specialized literature on harmonic retrieval in the case of vector signals, when the additive noise is colored, is still far from offering satisfactory solutions. For this class of signals, on the other hand, subspace system identification appears as a natural choice.
However it is well-known that stationary random processes with periodic components are not ergodic. Non-ergodicity means in particular that the limit when the sample size goes to infinity of the process sample covariance is sample dependent. In particular, the limit sample covariance depends on the random amplitudes of its elementary oscillatory components; see, e.g. Söderström and Stoica (1989, pp. 105–109). On the other hand, the asymptotic statistical properties of subspace methods (and, more generally, of correlation-based methods) depend essentially on the limit sample covariances, which in the presence of oscillatory or quasi-periodic components are not equal to the ensemble averages; i.e., do not coincide with the true covariances. Since parameter estimation procedures based on correlation methods require solving linear relations involving estimated sample covariances, a natural question to ask is if the parameter estimates obtained by solving these linear equations are consistent. This is generally true for signals which are second-order ergodic but sample dependence casts doubts on the validity of standard asymptotic statistical properties, like consistency, of subspace methods in this setting. In particular legitimate doubts arise on the validity of the standard proofs of consistency of subspace methods for signals of this type.
Sections 4 Subspace identification as partial realization, 5 Proof of consistency deal with the question of asymptotically recovering the system parameters (modulo similarity) starting from finite data by a standard subspace algorithm, formulated as an approximate partial realization problem. This setting permits to prove almost sure consistency of the algorithm without having to estimate the transient estimation errors inherent in the truncated least-squares regression approach of Peternell (1995), Peternell et al., 1995, Peternell et al., 1996.
Consistency of subspace methods for purely non-deterministic signals (time series) has been proved earlier in the just cited references. However, to the best of the authors’ knowledge, a proof of consistency when there are quasi-periodic components due to uncontrollable eigenvalues on the unit circle, does not exist. The only paper which comes close in spirit to what concerns us here is Bissacco, Chiuso, and Soatto (2007). In this paper however consistency analysis had to be left out as being “beyond the scope of the paper”. Finally, note that processes described by systems whose eigenvalues of modulus one are reachable for the driving process noise, do not concern us here as these processes are actually non-stationary and do not contain almost-periodic oscillations.
Section snippets
Stationary processes with an almost-periodic component
All random variables/vectors, denoted by lowercase boldface characters, will have zero mean and finite second order moments. The symbol denotes mathematical expectation. All random processes will be discrete time. It is a well-known fact that every vector-valued, say -dimensional, second-order stationary process admits an orthogonal decomposition where and are the purely deterministic (p.d.) and the purely non-deterministic (p.n.d.) components, the latter with an
Asymptotic covariance matching
Assume that we are in an ideal situation of observing an infinitely long sample trajectory of the output process of a true system of the form (2.2). From these data we form a string of limit sample covariances, where the Cesàro limits exist almost surely and are described in Proposition 1. We assume that the integer is chosen large enough so that .
Our goal will be to show that notwithstanding the limit p.d. covariance is sample
Subspace identification as partial realization
In this section, we shall recall the basic steps of a subspace identification algorithm for time series which will be generically referred to as CCA (Canonical Correlation Analysis) algorithm. CCA is actually a first step common to many subspace algorithms to obtain a factorization of the sample Hankel matrix and simultaneously accomplish order estimation. From this factorization some procedures compute estimates of the matrices by (approximate) partial realization by solving certain
Proof of consistency
Since the covariance function can be parametrized by a whole family of minimal matrix triplets mutually equivalent modulo similarity, a sequence of minimal estimates is called consistent if there is a sequence of nonsingular matrices such that converges for to a triplet realizing the true covariance of the system which generates the data. We shall call this convergence modulo similarity for short. An equivalent definition can be
Conclusions
In this paper, we have presented a general proof of strong consistency of subspace identification applied to signals with oscillatory components. Even if these signals are not ergodic and hence the standard consistency arguments based on second-order ergodicity do not apply, their sample covariance converges almost surely (to a sample dependent limit) and this fact can be exploited to show convergence of the estimates of the identifiable system parameters.
Acknowledgments
We are grateful to the associate editor and the referees for their scrupulous reading and for pointing out several errors in earlier versions of this manuscript.
Martina Favaro received a Laurea degree in Computer Engineering from the University of Padova, Italy, in 2008. After graduation she had a short time employment as a research assistant with the Department of Information Engineering, University of Padova and was granted an Erasmus scholarship for participation to the International doctoral school: Measuring, modelling and simulation of nonlinear dynamic systems, Department ELEC, Free University of Brussels, Belgium.
She is currently a Ph.D.
References (23)
Canonical correlation analysis of time series and the use of an information criterion
Asymptotic properties of subspace estimators
Automatica
(2005)- et al.
Analysis of the asymptotic properties of the moesp type of subspace algorithms
Automatica
(2000) - et al.
On the ill-conditioning of subspace identification with inputs
Automatica
(2004) - et al.
Canonical correlation analysis, approximate covariance extension and identification of stationary time series
Automatica
(1996) - et al.
Subspace algorithms for the stochastic identification problem
Automatica
(1993) The inverse problem of stationary covariance generation
Journal of Statistical Physics
(1969)State space modelling of time-series
(1990)- et al.
Classification and recognition of dynamical models
IEEE Transactions on Pattern Analysis and Machine Intelligence
(2007) - et al.
Matrix computations
(1983)
The statistical theory of linear systems
Cited by (0)
Martina Favaro received a Laurea degree in Computer Engineering from the University of Padova, Italy, in 2008. After graduation she had a short time employment as a research assistant with the Department of Information Engineering, University of Padova and was granted an Erasmus scholarship for participation to the International doctoral school: Measuring, modelling and simulation of nonlinear dynamic systems, Department ELEC, Free University of Brussels, Belgium.
She is currently a Ph.D. student with the Department of Computer Science Engineering and Control Systems, University of Brescia, Italy. Her research interests are in the area of system identification and signal processing.
Giorgio Picci is full professor with the University of Padova, Italy, Department of Information Engineering, since 1980. He graduated (cum laude) from the University of Padova in 1967 and since then has held several long-term visiting appointments with various American, Japanese and European universities among which Brown University, M.I.T., the University of Kentucky, Arizona State University, the Center for Mathematics and Computer Sciences (C.W.I.) in Amsterdam, the Royal Institute of Technology, Stockholm Sweden, Kyoto University and Washington University in St. Louis, Mo. He has been contributing to Systems and Control theory mostly in the area of modeling, estimation and identification of stochastic systems and published over 100 papers and edited three books in this area. He has been involved in various joint research projects with industry and state agencies. He has been chairman of the IFAC Technical Committee on Stochastic Systems, past member of the EUCA council, project manager of the Italian team for the Commission of the European Communities Network of Excellence System Identification (ERNSI) and general coordinator of the Commission of European Communities IST project RECSYS, in the fifth Framework Program. Giorgio Picci is a Life Fellow of the IEEE, an IFAC Fellow and a foreign member of the Swedish Royal Academy of Engineering Sciences.
- ☆
This work was supported in part by the project New methods and algorithms for Bayesian estimation, Identification and Adaptive Control funded by the Italian Ministry of higher education (MIUR). The material in this paper was not presented at any conference. This paper was recommended for publication in revised form by Associate Editor Wolfgang Scherrer under the direction of Editor Torsten Söderström.
- 1
Tel.: +39 049 8277705; fax: +39 049 8277614.