In this paper, we describe a method for transient stochastic analysis and apply it to develop a stochastic model for a true random number generator (RNG) circuit using intrinsic circuit noise. We use numerical simulation of stochastic differential equations to obtain time-domain transient analysis of the circuit. The simulation shows similar stochastic behavior and probability tuning as that observed in measurements of the fabricated chips. We further develop a small signal stochastic model of the circuit. The model reveals the role each device plays in contributing to overall stochastic behavior. The model and the simulation allow us to predict how the device parameters affect the performance. The method we propose here can be applied to other circuits where stochastic sample paths and ensemble statistics are necessary to characterize the circuits. Traditional noise analysis in the frequency domain is not adequate to provide this information.