Fine-granularity-scalability (FGS) has recently been standardized in MPEG-4 due to its flexibility in adapting in real-time to Internet bandwidth variations and its resilience to packet-losses. However, the flexibility and robustness come at the expense of degrading video quality when compared with non-scalable MPEG-4 video coding at a given bit-rate. To reduce this visual quality penalty at low and medium bit-rates, the "Frequency Weighting" (FW) method has been standardized that allows the prioritized transmission of "low frequency" DCT coefficients. In this paper, we propose a novel scene-characteristic-dependent adaptive FW method aimed at improving the visual quality of FGS. After a thorough analysis of the FGS (i.e., SNR) residual signal at various bit-rates, we conclude that for an improved subjective quality, different FW matrices should be used to improve the FGS visual quality depending on the video sequence characteristics. Subsequently, a simple classification mechanism is developed that categorizes the video sequences based on their brightness, motion and texture activity in four distinct classes, each using a different FW matrix. For each class, the appropriate FW matrix was determined a priori based on the differences of the residual signals for two representative single-layer bit-rates. This adaptive FW (AFW) method has been subjectively evaluated and shows a clear improvement in visual quality compared with non-frequency weighted or non-adaptive frequency weighted sequences.