This paper presents some theoretical results on the global exponential stability of recurrent neural networks with pure time-varying delays. It is shown that the recurrent neural network is globally exponentially stable, if the pure time-varying delays satisfy some limitations. In addition to providing new criteria for recurrent neural networks with pure time varying delays, these stability conditions also improve upon the existing ones with constant time delays and without time delays. Furthermore, it is convenient to estimate the exponential convergence rates of the neural networks by using the results.