In this paper, we elaborate on denoising schemes based on lossy compression. First, we provide an alternative interpretation of the so-called Occam filter and relate it with the complexity-regularized denoising schemes in the literature. Next, we discuss about the 'critical distortion' of a noisy source and argue that optimal denoising is achieved at the corresponding critical encoding rate rather than at the encoding rates suggested by other compression-based denoisers. Finally, we discuss the so-called 'indirect rate distortion problem'. We focus particularly on the high bit-rate encoding of noisy sources and show lossless compression of a denoised source is often very wasteful of bits, and suggest a simple way of determining an appropriate bit-rate for compressing a denoised source economically while retaining its initial denoised quality.