It is now common to access mobile video services through various Internet-connectable devices such as game stations, smartphones and tablets. This paper proposes a new energy management scheme for mobile devices by making use of the temporal scalability available in scalable video coding (SVC) techniques. The proposed scheme determines a quality index for each video segment, based on the pixel-wise difference between the luminance planes of consecutive frames at each frame-rate, and predicts the computation needed to decode each frame using data relating frame sizes to previous decoding times. A new dynamic voltage and frequency scaling (DVFS) scheme is then invoked to achieve the specified quality index. The proposed scheme was implemented on a smartphone and a user test was conducted to assess the acceptability of different degree of frame-rate reduction. Real measurements showed that the proposed scheme uses between 5% and 41% less energy than the Linux DVFS governor used in many smartphones, while maintaining acceptable video quality.