This paper presents a fast variable block size selection method for inter macro-blocks in H.264 video downsizing transcoding. It makes use of the relationship between motion direction and edge direction of macro-blocks, and selects the optimal inter mode by measuring the dominant edge direction using the residual macro-block. Without fully decoding the compressed high resolution video, Discrete Cosine Transform coefficients were extracted and reused in transform domain to construct the residual macro-block. Based on the measurement of dominant edge direction, only a small number of inter-modes are required for Rate Distortion Optimal evaluation. Experimental results show that the proposed method can reduce the computational complexity dramatically with little compression performance degradation.