An approach to user-assisted video object extraction at pixel-level accuracy is proposed. This approach is based on a novel framework that aims at reducing users' workload while making full use of user interaction. The framework takes over multipass scan on a video sequence and bidirectional tracking on the sequence's sub-shots during each pass. Systems based on this framework can not only free users from laborious work, but also be allowed to embed various complex but effective tracking algorithms. To fulfill a system, we propose an extended intelligent scissors for machine to precisely track VOs. The approach to tracking is a multilevel process, locating VO boundaries from a coarse level to a fine one and yielding pixel-accurate results of extraction. Experiments on MPEG test sequences show that this approach together with the proposed framework performs well in practice.