Recently, the MPEG-4 committee has approved the MPEG-4 Streaming Video Profile (SVP) that includes the Fine Granularity Scalability (FGS) as a new coding tool. In this paper, we propose novel techniques to further improve the temporal prediction schemes at the enhancement layer so that both the coding efficiency and error resilience are superior to the existing FGS while the fine granularity scalability is preserved. Our approach utilizes two parameters, the number of bitplanes /spl beta/(0/spl les/ /spl beta//spl les/maximal number of bitplanes) and the amount of predictive leak /spl alpha/ (0/spl les//spl alpha//spl les/1), to control the construction of the reference frame at the enhancement layer. These parameters /spl alpha/ and /spl beta/ can be temporally selected to provide tradeoffs between coding efficiency, error propagation and the predictive drift. It also encompasses several well-known FGS techniques as special cases for particular sets of /spl alpha/ and /spl beta/. The experimental results show over 2 dB improvement in coding efficiency using the MPEG-4 testing conditions.