Single ray tracking algorithm may cause significant performance deterioration in millimeter-wave (mmWave) channels with a cluster of indistinguishable multiple paths, especially in multiuser scenarios. By expressing the clustered channel vector as the Hadamard product of a time-varying cluster central path channel vector (CCPCV) and an intra-cluster structure vector (ICSV), an algorithm of jointly tracking the cluster central path and the ICSV is proposed in this letter. Numerical results show that our algorithm can substantially reduce the tracking error and improve the spectral efficiency compared with existing tracking methods.