An efficient depth map recovery algorithm, using concurrent object boundaries in texture and depth signals, is proposed in this work. We first analyze the effects of a distorted depth map on the qualities of synthesized views. Based on the analysis, we propose an object boundary detection scheme to restore sharp boundaries from a distorted depth map. Specifically, we initially estimate object boundaries from a depth map using the gradient magnitude at each pixel. We then multiply the gradient magnitudes of texture and depth pixels. Then, we suppress boundary pixels with non-maximum magnitudes and refine the object boundaries. Finally, we filter depth pixels along the gradient orientations using a median filter. Experimental results show that the proposed algorithm significantly improves the qualities of synthesized views, as compared with conventional algorithms.