Vehicle Trajectory Prediction (VTP) is one of the key issues in the field of autonomous driving. In recent years, more researchers have tried applying Deep Learning methods and techniques to VTP tasks. However, due to the black-box nature of Deep Learning, it cannot meet the interpretability and safety requirements of autonomous driving systems. Researchers have tried alleviating this problem by introducing driving knowledge in Deep Learning-based VTP. From the perspective of introducing driving knowledge, this paper systematically investigates the research status of DL-based VTP. First of all, this paper summarizes the research on VTP under three different problem formulations; secondly, this paper summarizes the application methods and application stages of driving knowledge in DL-based VTP; finally, this paper investigates and analyzes the VTP datasets and evaluation, and summarizes the knowledge contained in the datasets and its usage. Through the investigation and summary of problem formulation, knowledge usage, datasets, and evaluation of DL-based VTP, this paper analyzes the challenges and open questions of existing VTP research. It puts forward an outlook on future research directions.