To improve the system capacity of conventional orthogonal frequency division multiplexing (OFDM) systems, the layered modulation OFDM scheme is proposed. The modulated OFDM symbols are separated into two layers as differential modulated symbols and coherent symbols. The differential symbols are used as pilots, which are combined with the coherent modulated data and sent out by the transmitter. At the receiver side, the data in pilot position will be recovered after the differential demodulation. The recovered differential symbols will be used for channel estimation to demodulate the coherent symbols. Furthermore, the differential modulated data can employ low reception threshold and low complexity demodulator, which will facilitate the usage in mobile TV transmission. Simulations show the two layers' reception can be compatible with each other.