In this paper, the stability of a class of linear systems with stochastic parametric uncertainties is investigated. It is assumed that some parameters in the state matrices are not known precisely, but their distributions can be obtained. Such kind of stochastic parametric uncertainties are believed to be quite common in practice, and pose a significant challenge in design and analysis. This paper aims to identify conditions under which the system is stable in a stochastic sense. Our basic idea is to leverage on the recent developments on generalized Polynomial Chaos expansion theory, and transform the original stochastic system into a deterministic system of infinite order. Then, the stability of the original stochastic system can be implied from the stability of the infinite dimensional deterministic system, which can then be analyzed using Lyapunov function approaches existing in the literature. It is shown that the stability conditions depend on both the dynamics of the original system and the distribution of the stochastic parameters. To provide more insights into the obtained conditions, a special case where the system parameters are linear in the random variable is studied further. Numerical examples for uniformly distributed random variables are given to illustrate the results.