MPEG-4 has recently adopted the fine granularity scalability (FGS) video coding technology which enables easy and flexible adaptation to bandwidth fluctuations and device capabilities. Encryption for FGS should preserve such adaptation capabilities and allow intermediate stages in the delivery to process the media on the ciphertext directly. In this paper, we propose a novel scalable access control scheme with this property for the MPEG-4 FGS format. It offers free browsing of the low-quality base layer video but controls the access to the enhancement layer at different service levels based on either PSNR or bitrates. Both types of service levels are supported simultaneously without jeopardizing each other's security. The scheme is fast and degrades neither compression efficiency nor error resilience of the MPEG-4 FGS. The approach is also applicable to other scalable multimedia.