The large-scale antenna array is one key technology for the next generation wireless communications. Both spectral-efficiency and energy-efficiency can be improved significantly with appropriate beamforming which focuses signals on the target receivers in a large-scale antenna system (LSAS). However, as the formed beams become narrower, inaccurate channel state information (CSI) can lead to severe system performance degradation. The transmission rate selection is also important to enable full system capacity. Therefore, it is worth investigating the joint rate and beamwidth adaptation for data transmission in the presence of CSI uncertainties. In this paper, the dynamic selection of the transmission rate and the beamwidth is first modeled as a stochastic multi-armed bandit (MAB) problem with constraints. We then propose an iterative algorithm based on the upper confidence bound to adapt the rate and beamwidth jointly to get high throughput while providing long-term QoS guarantees. Numerical results demonstrate the benefits of taking beamwidth adaptation into account and illustrate the superior performance of our proposed algorithm.