Affected by acoustic transmission loss (TL), angular responses of sediments, time varying gain (TVG) residuals, and so on, side-scan sonar (SSS) images have the problem of gray attenuation perpendicular to the track direction. This problem seriously affects the subsequent stitching, detection, or other tasks. In this case, it is necessary to perform gray scale correction. The existing methods have a good correction effect on flat terrain but do not consider SSS images with rugged terrain. This article presents a gray scale correction method for SSS images with rugged terrain based on compensating acoustic attenuation. First, the influence of TL and the incident angle is analyzed, and a multiplicative attenuation model is established. Then, the image is segmented along the attenuation direction, and the segmentation intensity is calculated by using the multiplicative model for the echo points. Finally, the maximum segmentation intensity is taken as the target intensity, and the nonshaded points of the image are compensated and corrected. The experimental results show that our method (OURS) ensures the gray consistency of flat terrain on SSS images with rugged seafloor, highlights the seabed topography, and significantly improves the effect of image mosaic.