In-air handwriting necessitates consistent motion tracking, in contrast to millimeter-wave (mmWave) radar-based simple gesture recognition techniques. However, during long-duration gesture tracking, challenges, such as body motion interference and environmental clutter, become more pressing. Moreover, due to the lack of a supporting surface in in-air handwriting, slight arm tremors also can result in unsmooth trajectories. To address these challenges, this article proposes a two-stage processing framework called MMHTSR. In the first stage, the state-space equations are reestablished, and a locally correlated 2-D Gaussian process regression (GPR) algorithm is employed for interframe prediction. By incorporating uncertainty estimation, weights are assigned to the next frame data, effectively suppressing interference from nongestural targets. In the second stage, real-time smoothing and tracking of gesture trajectories are accomplished using a Kalman filter, followed by mapping the trajectories onto the Cartesian coordinate system. Finally, an end-to-end signal processing framework is deployed on a low-cost 60-GHz mmWave radar prototype, and gesture trajectory recognition is achieved using deep learning methods. Experimental results demonstrate that MMHTSR can accurately track motion gestures within the range of approximately 5–40 cm and successfully recognize 30 classes of in-air gesture trajectories, including uppercase letters A–Z and four interactive gesture actions. Furthermore, the proposed framework exhibits robust performance across various scenarios which shows its adaptability.