In recent years, with the rapid development of deep learning, single image super-resolution based on convolution neural network has achieved extensive research. However, most CNN-based method has difficulty in training and obtaining high quality images for large scale factors. To address these issues, we propose a network, which reconstructs HR images at large factors by progressively performing 2× SR on the input from the previous level. At each level, cascaded residual multi-scale aggregation blocks are used. The U-residual unit in it makes network simplifier and training easier without performance degradation. The multi-scale dilated unit in it provides more comprehensive information for image reconstruction. Before upsampling, the channel attention mechanism is adopted to recalibrate features. We train the network with two-stage training strategy which could accelerate the convergence and achieve better performance. Experiment results show that our proposed method is superior to the state-of-the-art methods on most datasets, especially on Urban100.