The electrocardiogram (ECG) signal is important for diagnosing cardiovascular diseases. However, in realistic scenario, the measured ECG signal is prone to be interfered by the artifacts caused from the respiration and the movement of patients. This artifact is called baseline wandering or baseline drifting and will lead to misdiagnosis if it is severe. Thus, pre-processing the measured ECG signal is necessary to make correct diagnosis. In this paper, we proposed a robust pre-processing method for extracting the baseline of ECG signals by the gradient varying weighting function. Our approach is adaptive to the input signal and is able to preserve the features of the ECG signal precisely. Simulation results show that our method outperforms other frequently used baseline extraction methods and has a good performance even if the input ECG signal is severely interfered by baseline drifting.