Activity recognition based on skeletons has drawn a lot of attention due to its wide applications in human-computer interaction, surveillance system. Compare with image data, a skeleton has a benefit of the robustness with background changing and computing efficiently dues to its low dimensional representation. With the rise of deep neural networks, a lot of works has been applied using both CNN and LSTM networks to solve this problem. In this paper, we proposed a framework for action recognition using skeleton data and evaluate it with different network architectures. We first modify the feature representation by adding motion information to a skeleton image, which gives useful information to the networks. After that, different networks architectures have been employed and evaluated to give insight into how well it will perform on this kind of data. Finally, we evaluated the system on two public datasets NTU-RGB+D and CMDFall to show the efficiency and feasibility of the system. The proposed method achieves 76.8% and 45.23% on NTU-RGB+D and CMDFall, respectively, which is competitive results.