With the better Internet infrastructure nowadays, the demand for delivering streaming or live video over IP-network via either unicast or multicast is getting higher. While packet loss is inevitable under heterogeneous best-effort IP networks, it is highly desirable to be able to control the dropping behavior within routers along the transmission path to achieve the best video quality at receivers. We propose a packetization scheme for baseline MPEG-4 fine granularity scalable (FGS) system as well as a novel queue management for routers, i.e., selective drop mechanism, to allow a router intelligently drop the less important packets at congestion while keeping the fairness between traffic of different transmission protocols.