This paper considers the problem of compressively sampling wide sense stationary random vectors with a low rank Toeplitz correlation matrix. A new structured deterministic sampling method known as the “Generalized Nested Sampling” (GNS) is used to fully exploit the inherent redundancy of low rank Toeplitz matrices. For a Toeplitz matrix of size N ×N with rank r, this sampling scheme uses only O(√r) measurements and allows exact recovery from noiseless measurements. This compression factor is independent of N and is shown to be larger than that achieved by existing random sampling based techniques for compressing Toeplitz matrices. The recovery procedure exploits the connection between Toeplitz matrices and linear prediction.