This paper considers the problem of system identification for linear systems. We propose a new system realization approach that uses an "information-state" as the state vector, where the "information-state" is composed of a finite number of past inputs and outputs. The system identification algorithm uses input-output data to fit an autoregressive moving average model (ARMA) to represent the current output in terms of finite past inputs and outputs. This information-state-based approach allows us to directly realize a state-space model using the estimated ARMA or time-varying ARMA parameters for linear time-invariant (LTI) or linear time-varying (LTV) systems, respectively. The paper develops the theoretical foundation for using ARMA parameters-based system representation using only the concept of linear observability, details the reasoning for exact output modeling using only the finite history, and shows that there is no need to separate the free and the forced response for identification. The proposed approach is tested on various different systems, and the performance is compared with state-of-the-art system identification techniques.