With the information from labeled RGB image an unsupervised method based on label transfer technology is proposed for 3D object recognition and segmentation in RGB-D images. We first use scale invariant features extracted from color space to retrieve a set of nearest neighbors of the input image from the labeled image database. Based on the projection matrix between the labeled image and the input image, the labels of the pixels in the labeled image are transferred to input image. And then a segmentation model and a clustering algorithm based on the geometric characteristics are designed to obtain the spatial and semantic consistent object regions in the RGB-D images. Compared to supervised object recognition, our method does not need to train a classifier using a lot of training images.