It is needed to evaluate how well the robot recognizes its environments while it is moving along the path to perform intended tasks. In general, the behavioral patterns of robot when it encounters sudden obstacles in a given environment may depend upon the obstacle avoidance algorithms that are programmed into the robot. In order to evaluate the performance of the environment recognition, we constructed the video image database sets which are photographed by both color image and depth image. We showed that the proposed performance evaluation method could be utilized effectively in evaluating the recognition performance of a robot.