In this paper, as a basic study for combination of turbo equalizer and 4D-8PSK-TCM (4 Dimensional 8 Phase Shift Keying Trellis Coded Modulation) system, which is recommended for X-band band satellite communication in CCSDS, a system combining 8PSK-TCM system and turbo equalizer has been designed and performance evaluation of the system has been performed. BER performance of 8PSK and 8PSK-TCM systems has evaluated and analyzed in AWGN, Proakis B and Proakis C channels in detail. As results of simulation, BER performance of 8PSK-TCM system is better than 8PSK system in AWGN environment. Furthermore, it can be confirmed that the system combining the 8PSK-TCM system with the turbo equalizer can further improve the BER performance through iterative equalization in the ISI channel environment. The 8PSK-TCM system combined with the turbo equalizer designed in this paper has BER performance of 3×10^-4 under 8dB SNR in Proakis B with 5 iterative equalization and BER performance of 1.5×10^-3 under 10dB SNR in Proakis C with 5 iterative equalization.