Recently, sports and ICT technology have been combined, enabling quantitative and objective analysis of sports and athlete competence. In the case of soccer, quantitative analysis of competition and athletes is underway in various companies, but due to technical limitations, many data are still being generated based on the manual work of experts. In this paper, we propose an object motion recognition technique which is a basis for further automation of soccer analysis. We first classify objects in soccer game and define recognizable motion for each object category. After that, we design 3D CNN with spatiotemporal characteristics and extract the motion information that each object is currently taking from the match video. As can be seen from the experimental results, it can be confirmed that the proposed technique not only has higher speed performance than the existing methods, but also has high accuracy. In addition, it can be confirmed that there is a high possibility of expanding to areas such as CCTV surveillance.