In this paper, we present a robust motion-based object detection system that corrects for the motion of an unstable camera. Assuming that the global camera motion may be modeled as an affine transform of the image between two successive frames, the proposed method is able to correct for camera motion using an elastic registration algorithm (ER). The local motion is then estimated from a current image and affine-transformed previous image. Finally, object regions are detected using the estimated local motions. Experimental results show that the proposed system is able to robustly detect moving objects in unstable imaging environment for consumer surveillance systems.