Linear analog coding, or, transformation through linear analog matrices, exhibits interesting relation to space-time codes and modulation diversity, and finds useful application in orthogonal frequency division multiplexing (OFDM). This paper analyzes analog codes, establishes their performance limits in terms of the mean square error (MSE), and identifies the best practices. Two optimal detectors, linear minimum mean square error (LMMSE) detector and maximum likelihood (ML) detector, are developed and analyzed, and their performance lower bounds are computed, respectively. It is shown that LMMSE decoder generally outperforms ML decoders (under the MSE criterion), but the gain diminishes as the signal-to-noise ratio increases. It is further shown that the choice of the analog code (i.e. the analog matrix) makes a considerable difference under ML detection, but not so much under LMMSE detection. Finally, the unitary codes, an important class of analog codes whose subsets form discrete cosine transform (DCT) codes, discrete Fourier transform (DFT) codes, BCH analog codes and Reed-Solomon analog codes, are established as the best class of linear analog codes, as they achieve both bounds simultaneously.