In this paper, we propose a novel scalable video coding algorithm based on adaptively weighted multiple reference frame method. To improve the coding efficiency in the enhancement layer, double reference picture and double motion vector algorithms are employed for motion compensated prediction. Furthermore, an enhancement layer frame is predicted from the adaptively weighted superposition of the corresponding base layer frame and the double motion compensated frames, according to the the input video characteristics and the feedback channel condition. By employing an adaptive reference selection scheme at the decoder, the proposed method can reduce the drift problem significantly. Experimental results show that the proposed algorithm yields higher PSNR performance than H.263+ both in error-free and severe packet loss channel conditions.