This paper studies the impact of packet loss on H.264 scalable video coding (SVC). A Markov Chain (MC) with 2^N states is developed to describe the error propagation process inside a group of pictures (GOP). The characteristic of different packet loss events is captured by the initial state vector of the MC. Based on the proposed model, the performance of different GOP structures can be evaluated under both random and burst packet losses.