Hyperspectral image (HSI) is usually polluted by dense additive white Gaussian noise (AWGN). The noise intensity of AWGN is different for different bands of the HSI, and it has been utilized by many denoising algorithms. However, many of them did not convey this phenomenon in the objective function. In this paper, a bandwise noise model is proposed to overcome the problem. Furthermore, based on the low-rank structure of the HSI, the proposed bandwise noise model is combined with the low-rank matrix factorization to obtain a new efficient HSI denoising algorithm. Additionally, as the noise estimation is an important preprocessing step for the proposed algorithm, a classical noise estimation algorithm is improved to increase its efficiency. The proposed algorithm is used on the HSI corrupted by mixed noise. Both simulated and real experiments demonstrate the state-of-the-art performance and efficiency of the proposed algorithm.