In this paper, infinite and finite sample properties of set membership identification are investigated in a stochastic setting. In particular, the size of the membership set in the presence of not only disturbance but also parameter uncertainty is estimated. The bounds of the disturbance and the parameter uncertainty are assumed to be tight as well as known, where tight means that the disturbance and the parameter uncertainty take a value around their extreme points with nonzero probability. The following results are obtained. (i) Infinite sample case: The size of the membership set converges to zero with probability one as the number of samples tends to infinity if the regressor is persistently exciting and the bounds of the disturbance and the parameter uncertainty are tight. This means that the membership set converges to the true but unknown parameter. (ii) Finite sample case: For a given number of samples, the size of the membership set can be estimated with a probabilistic confidence if the regressor is periodic and persistently exciting, and the bounds of the disturbance and the parameter uncertainty are tight. This result also clarifies the necessary number of samples such that the size of the membership set is less than a specified bound with a specified probability.