In recent years, deep convolutional neural networks (CNNs) have achieved great success in Single Image Super-Resolution (SISR). Most existing networks for Super-Resolution (SR) concentrate on wider or deeper network designs, leading to neglect of the feature correlations of intermediate layers. In this letter, a novel Multilevel and Multiscale Network for SISR (M2SR) is presented. The proposed network framework consists of four parts, including the feature extraction network, the cascade residual U-shaped blocks, the channel-wise attention U-shaped block and the fusion reconstruction network. Specially, the residual U-shaped blocks are designed to extract different scales of features, which are stacked to better refine the multifeatures. Then, to fully exploit the different levels of features, a channel-wise attention U-shaped block (At-U) is proposed to adjust the feature weights, which can adaptively enhance the feature expression and correlation learning. Finally, a fusion reconstruction network is constructed to fuse the different scales of the enhanced features to achieve the reconstructed result. Quantitative and qualitative evaluations of four public datasets show that the proposed method can achieve better performance compared with the state-of-the-art SR methods.