With the great popularity and extensive application of Kinect, the Internet is sharing more and more depth data. To effectively use plenty of depth data would make great sense. In this paper, we propose a generalized dynamic depth data matching framework for action retrieval. Firstly we focus on single depth image matching utilizing both depth and shape feature. The depth feature used in our method is straightforward but proved to be very effective and robust for distinguishing various human actions. Then, we adopt shape context, which is widely used in shape matching, in order to strengthen the robustness of our matching strategy. Finally, we utilize Dynamic Time Warping to measure temporal similarity between two depth video sequences. Experiments based on a dataset of 17 classes of actions from 10 different individuals demonstrate the effectiveness and robustness of our proposed matching strategy.