Recently, a common and popular method that produces competitive accuracy is to employ dense trajectories to identity human action. However, computing descriptors of dense trajectories may spend lots of time, and many trajectories which belong to the background trajectories may not be useful for the recognition. Moreover, the relationship between trajectories is always ignored. In this paper, we propose a trajectories-based motion neighborhood feature (TMNF) method for action recognition. We first select the trajectories of central particular region at the original video resolution to reduce the computation as well as the background trajectories. A new descriptor, which is referred to as TMNF, is proposed to explore the orientation and motion relationship between different trajectories. Finally, an improved vector of locally aggregated descriptors (IVLAD) method is used to represent videos and linear SVM is applied for classification. Experiments on the YouTube dataset demonstrate that our approach achieves superior performance.