Channel estimation is crucial to modern wireless systems and becomes increasingly challenging when the ultra-sized antenna is configured in sub-6GHz wireless communication systems. In an ultra-massive multiple-input multiple-output (U-MIMO) orthogonal frequency division multiplex (OFDM) system, the channel demonstrates spatial non-stationarity. Additionally, the limited pilot location in the OFDM system further complicates the channel estimation process. In this paper, we propose a model-data dual-driven (MDD) scheme to jointly perform the model-driven non-stationary channel denoising and the data-driven channel interpolation in an end-to-end way, which is followed by a low-complexity channel refinement module to improve the robustness of the proposed scheme. Specifically, image contour extraction (ICE) is utilized to effectively eliminate the non-stationary noises in the channel matrices before being sent to the downstream interpolation network. An enhanced convolutional neural network (CNN)-based residual network (eCNN-RN) is developed to perform non-linear interpolations for recovering the U-MIMO-OFDM channels. Based on ICE, the proposed online refinement module can improve the generalizability of the learned model to a practical environment. Numerical experiments demonstrate the efficiency and the effectiveness of the cross-fertilization of the model-driven and data-driven approaches.