The finite-impulse response (FIR) filter technique has been widely used for pre-emphasis of channels to mitigate the intersymbol interference (ISI) resulted from both frequency dependent losses and reflections. This paper proposes a systematic methodology, based on arbitrary step response, to determine the tap setting of multi-tap FIR filter for best eye diagram improvement. The required tap number and the optimal tap coefficients are determined according to the compensation efficiency and hence the ultimate performance of FIR filter is evaluated. Eventually, the compensation results for two specific 5Gbps signalling systems, which include significant effects of losses and multiple reflections are demonstrated to validate the optimization method.