Many video coding standards, including the state-of-the-art High Efficiency Video Coding (HEVC) standard, employ Motion Estimation (ME) at sub-pel precision level to improve the prediction accuracy. However, interpolation and additional Motion Vector (MV) search associated with sub-pel ME result in increased computational complexity. In this paper, an algorithm for fast ME is proposed based on neighbouring sample cost similarity. The costs corresponding to the neighbouring MVs around the optimal MV at different precision levels are analyzed and used to interpolate an error surface. The estimated location of the surface minimum is then used to either skip the remaining ME process, or to limit the number of points to test. Experimental evaluation on high resolution sequences shows that the proposed method can speed up the ME module by 55.5% for 1.5% BD rate losses. Furthermore, the proposed fast quarter-pel ME reduces the quarter-pel ME time by 79.6% while achieving only 17.6% of theoretical encoding losses.