Obstacle detection is one of the main components of the control system and navigation of autonomous vehicles. This paper presents a novel approach to obstacle detection and segmentation with a RGB-D sensor. Different from the traditional approaches which only detect whether there exist obstacles, our approach not only can detect the obstacles but also can distinguish between dynamic obstacles and static obstacles. Base on the information received by the kinect sensor, the mobile robot can choose different avoidance strategy when facing different kinds of obstacles intelligently. First, we get a 3D point cloud from the depth image and compute the height of each point from ground plane which is estimated during a calibration step. In this step, we can discriminate which point belongs to obstacles. Then we use a batch of depth images to get the dynamic objects of each image. Finally, the obstacle map is an orthographic projection of these obstacle points along the normal to the ground plane. Experimental results carried out with a mobile platform in an indoor environment have shown that the system can detect and segment stationary and moving obstacles.