This paper proposes a novel reflection removal method for RGB-D images that achieve reflection removal and depth map recovery simultaneously. In general, there is a strong structure correlation between an RGB image and a depth map in gradient domain. Based on this fact, we introduce a novel regularization for RGB-D images named the multi-modal structure tensor total variation (MSTV). A proposed minimization problem based on MSTV which is constructed by two minimization problems, reflection removal and depth map recovery, is solved by using alternating direction method of multipliers (ADMM). Experimental results show the effectiveness of our method by applying it to both artificial and real-world images.