Deep convolutional neural networks (CNNs) have demonstrated remarkable progress on single image super-resolution (SISR). However, most networks do not make full use of the rich prior information of the image itself, and get smoother results. To solve this problem, we propose a two-way guided super-resolution reconstruction network based on gradient prior (TWGSR), which uses gradient branches to achieve two-way guidance and employs multi-level gradient loss constraints to reconstruct high-resolution images with more textures. In addition, we propose a multi-scale module (MSM), which adopts dilated convolution to aggregate feature information of different scales, and we add an error feedback module (EFM) to compensate for sampling errors to refine the extracted feature maps. Furthermore, we propose an improved cross residual-in-residual dense block (CRRDB) to enhance the feature extraction capability of the module. Experimental results show that our TWGSR achieves favorable performance against state-of-the-art methods.