In this letter, we propose a machine learning-based vision-aided beam selection (ML-VBS) for millimeter-wave indoor multi-user communications. The proposed scheme is aimed at addressing the beam selection overhead with narrow beams in a multi-user scenario. The proposed scheme relies on a base station (BS) equipped with a single camera to observe the scene and estimates the angles to the multiple users. Given the estimated angle information and the limited number of radio frequency chains at the BS, two serial deep neural network structures are employed for joint user and beam selection subject to a minimum rate constraint. The numerical evaluation shows that the proposed ML-VBS scheme achieves a good performance in terms of the multi-user angle estimation, achievable sum rate and low computational complexity compared to conventional beam selection techniques.