This paper presents a single fisheye lens camera-based visual surveillance system for monitoring a wide area. A fisheye lens has a wider field-of-view (FOV) than normal lenses at the cost of a barrel distortion in the acquired image. After correcting the barrel distortion, the proposed algorithm detects objects, and performs tracking using a histogram-based Gaussian mixture model (GMM). Experimental results show that the proposed algorithm can efficiently detect objects by reducing the geometric distortion of the input image. For this reason it is suitable for not only surveillance cameras but also consumer applications of video object detection and recognition.