An adaptive image and video resizing algorithm based on the frequency domain analysis is proposed in this work. Given an image, we first construct an importance map by combining the gradient and the saliency information. We partition the image into several strips so that each strip contains pixels of similar importance levels. We model the distortion, which is caused by scaling a strip, in the frequency domain. Then, we scale each strip adaptively to minimize the overall distortion of the whole image. Moreover, we extend the proposed algorithm for video resizing. We add the motion term to construct the importance map, and suppress excessive parameter variations to achieve jitter-free video resizing. Simulation results show that the proposed algorithm provides higher quality resizing results than conventional algorithms, although it requires lower computational complexity.