This paper introduces a novel method for image classification based on both texture and depth information. The proposed method uses depth maps in order to improve on the performance of conventional texture-based classification. Depth features are extracted by capturing shapes of depth map slices. The extracted depth features are encoded in the form of sparse representation. Fusion of texture and depth lead to state-of-the-art performance in three-dimensional image classification.