Stereo matching consists of four steps: cost computation, cost aggregation, disparity optimization, and disparity refinement. While the first three steps have been actively researched, few efforts have been taken on disparity refinement. In this paper, we propose a novel segmentation-based disparity refinement method. The proposed method uses the property that the pixels belonging to the same segment have similar disparity values. Using this property, the proposed method can detect the error regions with various shapes and sizes. For the detected error regions, the proposed method applies the weighted median filter to remove the disparity error. In the experimental results, the proposed method improves the quality of disparity maps by increasing average bad pixel rate up to 17.5 % compared to the benchmark methods.