In Internet of Vehicles (IoV), vehicles communicate with Roadside Units (RSU) to ensure driving safety. The variability in complex road trajectories intensifies the angle changes between vehicles and RSU, impacting the stability of IoV Millimeter Wave (mmWave) communications. The paper proposes an Integrated Sensing and Communication (ISAC) RCG-Net beam tracking solution for IoV on complex road trajectories, integrating Convolutional Neural Networks (CNN) and Gated Recurrent Units (GRU). The RCG-Net beam tracking solution leverages the powerful feature learning capability of deep neural networks, predicting and tracking angles using spatial features of echo signals and historical information features. This enhances beam tracking precision and resolves communication instability issues in IoV on complex road trajectories. Simulation results demonstrate that the proposed solution achieves high angle tracking accuracy and communication performance on intricate road trajectories, outperforming beam tracking solutions based on communication feedback and state evolution model.