An algorithm for compensating the effects of backlight in images and videos is proposed in this work. We first determine the region of interest (ROI) to compensate mainly using a saliency map, which is based on darkness, skin color, color and texture prominency features. We then compute the compensating offset value for each pixel. Initial offset values are derived to improve the brightness and the contrast of the ROI, and also to provide temporally consistent output frames in case of the video compensation. Finally, we obtain the final offset values by minimizing an energy function, consisting of a data term and a smoothness term. Simulation results show that the proposed algorithm improves the picture qualities of backlit images and videos efficiently.