Skip to main content
Log in

When are two multivariate random processes indistinguishable

  • Published:
Journal of Combinatorial Optimization Aims and scope Submit manuscript

Abstract

Prediction error identification methods have been recently the objects of much study, and have wide applicability. The maximum likelihood (ML) identification methods for Gaussian models and the least squares prediction error method (LSPE) are special cases of the general approach. In this paper, we investigate conditions for distinguishability or identifiability of multivariate random processes, for both continuous and discrete observation time T. We consider stationary stochastic processes, for the ML and LSPE methods, and for large observation interval T, we resolve the identifiability question. Our analysis begins by considering stationary autoregressive moving average models, but the conclusions apply for general stationary, stable vector models. The limiting value for T → ∞ of the criterion function is evaluated, and it is viewed as a distance measure in the parameter space of the model.

The main new result of this paper is to specify the equivalence classes of stationary models that achieve the global minimization of the above distance measure, and hence to determine precisely the classes of models that are not identifiable from each other. The new conclusions are useful for parameterizing multivariate stationary models in system identification problems. Relationships to previously discovered identifiability conditions are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Akaike H (1973) Maximum likelihood identification of Gaussian autoregressive moving average models. Biometrika 60:255–265

    Article  MATH  MathSciNet  Google Scholar 

  • Astrom KJ, Soderstrom T (1974) Uniqueness of the maximum likelihood estimates of an ARMA model. IEEE Trans on Aut Control AC-19(6):769–773

    Article  MathSciNet  Google Scholar 

  • Baram Y (1978) A sufficient condition for consistent discrimination between stationary Gaussian models. IEEE Trans on Aut Control AC-23(5):958–960

    Article  Google Scholar 

  • Caines PE (1976) Prediction error identification methods for stationary stochastic processes. IEEE Trans on Aut Control AC-21:500–506

    Article  MathSciNet  Google Scholar 

  • Caines PE, Wei YJ (1988) Hierarchical hybrid control systems: A lattice theoretic formulation. IEEE Trans on Aut Control: Special Issue on Hybrid Systems 43(4)

  • Chan CK, Shynk JJ (1990) Stationary points of the constant modulus algorithm for real Gaussian signals. In IEEE Trans ASSP, ASSP-38:2176–2181

    Article  Google Scholar 

  • Hannan EJ (1970) Multiple Time Series, New York, Wiley

    MATH  Google Scholar 

  • Huber PJ (1981) Robust statistics, Wiley-Interscience

  • Kailath T (1980) A general likelihood ratio formula for random signals in Gaussian noise. IEEE Trans on Infor Theory AC-25(2):197–207

    Google Scholar 

  • Kashyap RL, Nashburg RE (1974) Parameter estimation in multivariate stochastic difference equations. IEEE Trans on Aut Control AC-19(6):784–797

    Article  Google Scholar 

  • Kashyap RL (1977) A Bayesian comparison of different classes of dynamic models using empirical data. IEEE Trans on Aut Control AC-22

  • Kazakos D, Papantoni-Kazakos P (1980) Spectral distance measures between Gaussian processes. IEEE Trans on Aut Control AC-25(5):950–959

    Article  MathSciNet  Google Scholar 

  • Kazakos D (1980) On resolution and exponential discrimination between Gaussian stationary vector processes and models. IEEE Trans on Aut Control AC-25(2)

  • Kazakos D (1982) Distance measures and estimation performance bounds for continuous time data. IEEE Trans on Infor Theory IT-28

  • Kosut RL, Lau MK, Boyd SP (1992) Set-membership identification of systems with parametric and nonparametric uncertainty. IEEE Trans on Aut Control 37(7):929–942

    Article  MathSciNet  MATH  Google Scholar 

  • Ljung L, Glover K (1981) Frequence domain versus time domain methods in system identification. Automatica 17(1):71–86

    Article  MathSciNet  MATH  Google Scholar 

  • Ljung L, Guo L (1997) Classical model validation for control design purposes. Math Modelling of Syst 3:27–42

    MathSciNet  MATH  Google Scholar 

  • Ljung L (1978) Convergence analysis of parametric identification methods. IEEE Trans on Aut Control AC-23(5):770–782

    Article  MathSciNet  Google Scholar 

  • Martin RD (1978) Asymptotic properties of M-estimates for p-th order autoregressions. Tech Report no 213, Dept of Electr Eng Univ of Washington, Seattle

    Google Scholar 

  • Masreliez CJ, Martin RD (1977) Robust Bayesian estimation for the linear model and robustifying the Kalman filters. IEEE Trans on Aut Control AC-22:361–373

    Article  MathSciNet  Google Scholar 

  • Mehra RK (1974) Optimal input signals for parameter estimation in dynamic systems–-Survey and new results. IEEE Trans on Aut Control AC-19(6):753–768

    Article  MathSciNet  Google Scholar 

  • Ninness B, Goodwin G (1994) Estimation of model quality In: Proc. 10th IFAC Symposium on System Identification, vol I, pp. 25–44

    Google Scholar 

  • Poolla K, Khargonekar PP, Tikku A, Krause J, Nagpal K (1994) A Time-domain approach to model validation. IEEE Trans on Aut Control AC-39:951–059

    Article  MathSciNet  Google Scholar 

  • Scheweppe FC (1965) Evaluation of likelihood functions for Gaussian signals. IEEE Trans on Infor Theory pp 61–70

  • Tissanen J, Caines PE (1974) Consistency of maximum likelihood estimators for ARMA processes. Tech Report 7424, Univ of Toronto, Control Syst Group of Elect Eng Dept

  • Tong L, Xu G, Kailath T (1994) Blind identification and equalization based on second-order statistics: A time-domain approach. IEEE Trans Inform Theory 40:340–350

    Article  Google Scholar 

  • Tse E, Anton J (1972) On the identifiability of parameters. IEEE Trans on Aut Control AC-17(5): 637–645

    Article  MathSciNet  Google Scholar 

  • Zang Z, Bitmead RR, Gevers M (1955) Iterative Weighted Least-squares Identification and Weighted LQC Control Design. Automatica 31(11):1577–1594

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Demetrios Kazakos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kazakos, D., Makki, S. When are two multivariate random processes indistinguishable. J Comb Optim 11, 263–278 (2006). https://doi.org/10.1007/s10878-006-7907-1

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10878-006-7907-1

Keywords

Navigation