One VLSI friendly fast motion estimation (ME) algorithm is proposed in this paper. Firstly, theoretical analysis shows that image rich of sharp edges and texture is regarded as high frequency abundant image and macroblocks (MBs) in such image will express large pixel difference. In our paper, we apply adaptive subsampling method during ME process based on pixel difference analysis, so the computation complexity of full pixel pattern can be reduced. Secondly, statistic analysis shows that for MBs with static feature, the ratio of selecting previous reference frame as best one is very high and multiple reference frame technique is not required for these MBs. Based on this analysis, we give out a block overlapping method to pick out static MBs and apply MRF elimination process. Thirdly, since many redundant search positions exist in MB with small motion trend and large search range is only contributive to MB with big motion, we extract motion feature after ME on first reference frame and use it to adjust search range for rest ME process. So, the computation complexity of redundant search positions is eliminated. Experimental results show that, compared with hardware friendly full search algorithm, our proposed algorithm can reduce 71.09% to 95.26% ME time with negligible video quality degradation. Moreover, our fast algorithm can be combined with existing fast ME algorithms like UMHexagon method for further reduction in complexity and it is friendly to hardware implementation.