Millimeter-wave (mmWave) communication is one of the key technologies to drive future high-capacity communication in the sixth generation (6G) network. However, the easily blocking characteristics of mmWave communication links hinder the reliable transmission in mmWave communication. To improve the reliability of mmWave transmission, we investigate the joint coordinated multi-beam selection and power control scheme that maximizes average sum rate of UEs. To efficiently solve this problem, we recast it as a potential game and propose a best response based approach to find the Nash equilibrium (NE). Simulation results have shown that the proposed solution has better average sum rate, and can achieve more reliable mmWave communications.