Recently, numerous salient object detection methods are proposed for different data types. And a reliable method, which can accurately extract complete salient objects, is beneficial to various vision tasks. However, existing methods may fail in highlighting the entire salient object uniformly. In this work, we propose a simple and universal framework aiming to improve the detection result of existing methods. To remove inaccurate salient regions, we apply location prior and adaptive de-noising to prior saliency maps extracted from existing methods in the pre-processing step. Then, an iteration optimization algorithm considering local smoothness and global similarity is introduced to refine the pre-processed saliency map. The experimental results show that the proposed framework can universally enhance the performance of state-of-the-art salient object detection methods for 2D, 3D and light field data.