Abstract
Prediction error identification methods have been recently the objects of much study, and have wide applicability. The maximum likelihood (ML) identification methods for Gaussian models and the least squares prediction error method (LSPE) are special cases of the general approach. In this paper, we investigate conditions for distinguishability or identifiability of multivariate random processes, for both continuous and discrete observation time T. We consider stationary stochastic processes, for the ML and LSPE methods, and for large observation interval T, we resolve the identifiability question. Our analysis begins by considering stationary autoregressive moving average models, but the conclusions apply for general stationary, stable vector models. The limiting value for T → ∞ of the criterion function is evaluated, and it is viewed as a distance measure in the parameter space of the model.
The main new result of this paper is to specify the equivalence classes of stationary models that achieve the global minimization of the above distance measure, and hence to determine precisely the classes of models that are not identifiable from each other. The new conclusions are useful for parameterizing multivariate stationary models in system identification problems. Relationships to previously discovered identifiability conditions are discussed.
Similar content being viewed by others
References
Akaike H (1973) Maximum likelihood identification of Gaussian autoregressive moving average models. Biometrika 60:255–265
Astrom KJ, Soderstrom T (1974) Uniqueness of the maximum likelihood estimates of an ARMA model. IEEE Trans on Aut Control AC-19(6):769–773
Baram Y (1978) A sufficient condition for consistent discrimination between stationary Gaussian models. IEEE Trans on Aut Control AC-23(5):958–960
Caines PE (1976) Prediction error identification methods for stationary stochastic processes. IEEE Trans on Aut Control AC-21:500–506
Caines PE, Wei YJ (1988) Hierarchical hybrid control systems: A lattice theoretic formulation. IEEE Trans on Aut Control: Special Issue on Hybrid Systems 43(4)
Chan CK, Shynk JJ (1990) Stationary points of the constant modulus algorithm for real Gaussian signals. In IEEE Trans ASSP, ASSP-38:2176–2181
Hannan EJ (1970) Multiple Time Series, New York, Wiley
Huber PJ (1981) Robust statistics, Wiley-Interscience
Kailath T (1980) A general likelihood ratio formula for random signals in Gaussian noise. IEEE Trans on Infor Theory AC-25(2):197–207
Kashyap RL, Nashburg RE (1974) Parameter estimation in multivariate stochastic difference equations. IEEE Trans on Aut Control AC-19(6):784–797
Kashyap RL (1977) A Bayesian comparison of different classes of dynamic models using empirical data. IEEE Trans on Aut Control AC-22
Kazakos D, Papantoni-Kazakos P (1980) Spectral distance measures between Gaussian processes. IEEE Trans on Aut Control AC-25(5):950–959
Kazakos D (1980) On resolution and exponential discrimination between Gaussian stationary vector processes and models. IEEE Trans on Aut Control AC-25(2)
Kazakos D (1982) Distance measures and estimation performance bounds for continuous time data. IEEE Trans on Infor Theory IT-28
Kosut RL, Lau MK, Boyd SP (1992) Set-membership identification of systems with parametric and nonparametric uncertainty. IEEE Trans on Aut Control 37(7):929–942
Ljung L, Glover K (1981) Frequence domain versus time domain methods in system identification. Automatica 17(1):71–86
Ljung L, Guo L (1997) Classical model validation for control design purposes. Math Modelling of Syst 3:27–42
Ljung L (1978) Convergence analysis of parametric identification methods. IEEE Trans on Aut Control AC-23(5):770–782
Martin RD (1978) Asymptotic properties of M-estimates for p-th order autoregressions. Tech Report no 213, Dept of Electr Eng Univ of Washington, Seattle
Masreliez CJ, Martin RD (1977) Robust Bayesian estimation for the linear model and robustifying the Kalman filters. IEEE Trans on Aut Control AC-22:361–373
Mehra RK (1974) Optimal input signals for parameter estimation in dynamic systems–-Survey and new results. IEEE Trans on Aut Control AC-19(6):753–768
Ninness B, Goodwin G (1994) Estimation of model quality In: Proc. 10th IFAC Symposium on System Identification, vol I, pp. 25–44
Poolla K, Khargonekar PP, Tikku A, Krause J, Nagpal K (1994) A Time-domain approach to model validation. IEEE Trans on Aut Control AC-39:951–059
Scheweppe FC (1965) Evaluation of likelihood functions for Gaussian signals. IEEE Trans on Infor Theory pp 61–70
Tissanen J, Caines PE (1974) Consistency of maximum likelihood estimators for ARMA processes. Tech Report 7424, Univ of Toronto, Control Syst Group of Elect Eng Dept
Tong L, Xu G, Kailath T (1994) Blind identification and equalization based on second-order statistics: A time-domain approach. IEEE Trans Inform Theory 40:340–350
Tse E, Anton J (1972) On the identifiability of parameters. IEEE Trans on Aut Control AC-17(5): 637–645
Zang Z, Bitmead RR, Gevers M (1955) Iterative Weighted Least-squares Identification and Weighted LQC Control Design. Automatica 31(11):1577–1594
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kazakos, D., Makki, S. When are two multivariate random processes indistinguishable. J Comb Optim 11, 263–278 (2006). https://doi.org/10.1007/s10878-006-7907-1
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s10878-006-7907-1