Hybrid analog/digital precoding provides an energy- efficient solution for multi-user millimeter-wave (mmWave) systems by utilizing a small number of radio frequency (RF) chains for digital precoding and a phase shifter (PS) network for analog precoding. However, the conventional fully-connected and partially-connected architectures exist the problem of terrible trade-off between performance and hardware complexity. In this paper, we propose a joint user scheduling and hybrid precoding scheme for multi-user mmWave systems with two-layer PS network at the base station. Specifically, we propose a rank-one approximation based hybrid precoding algorithm to handle the hardware constraints on analog precoder. Moreover, a vertical user grouping scheduling algorithm is proposed to improve the achievable-rate performance by simultaneously serving users with similar vertical angle of departures (AoDs) but different horizontal AoDs. Simulation results demonstrate that the proposed scheme approaches the performance of fully- connected architecture with much lower hardware complexity.