In this paper, Vulnerable Road User (VRU) trajectory prediction for autonomous driving based on the Intention-Attention-Gate Recurrent Unit (IA-GRU), Improved Social Force Model (ISFM) and Adaptive Boosting (AdaBoost) is systematically investigated. Firstly, a novel IA-GRU is proposed for VRU (pedestrian, cyclist, and electric cyclist) trajectory prediction. VRU intention (waiting/crossing), VRU heterogeneity (age and gender), VRU-VRU interactions and VRU-dynamic vehicle interactions are taken into account. Attention is used to obtain the influence weights of the above factors used for VRU trajectory prediction. Secondly, a micro-dynamic ISFM is developed for VRU trajectory prediction. The impact of zebra crossing, collision avoidance with vehicles and VRUs, and VRU heterogeneity are considered. Moreover, traffic data collected by an unmanned aerial vehicle (UAV) is obtained and analyzed, and the parameters of the ISFM are calibrated by the Maximum Likelihood Estimation (MLE). Finally, a data-driven integrated approach based on the IA-GRU and ISFM is proposed, and AdaBoost is used to prevent the model from overfitting and improve the prediction accuracy. The results indicate that the integrated model outperforms the existing methods, and the prediction accuracy is improved by more than 11% based on the collected traffic data, which can give us great confidence to use the integrated model in the autonomous driving domain to improve the safety of VRUs.