Unattended object detection is an important task in surveillance. Thus, we propose a new method to detect unattended object by modeling the objects as newly learned temporal background. We use edge-segments to model the structural changes in the scene. Specifically, we construct distributions of these edge-segments to analyze the scene, and to segment its different components: background, fore-ground, and the interesting new objects. Additionally, we propose a clustering algorithm to recover the unattended objects from a set of edges based on the assumption that spatially close edges come from the same object. Our experiments on several datasets validate our proposed method.