This paper proposes a new two-stream neural network which combines the traditional background modeling method with a deep learning network to detect moving objects. The input for the deep neural network is the original image and its corresponding foreground image, while the output is the bounding boxes of the moving objects in the image. Traditional CNN methods cannot distinguish moving objects from static objects, but the method in this paper successfully solves this problem.