Disparity matching and object extraction of stereoscopic video are critical procedures for many computer vision applications. Current research in this area is mostly focused on algorithm design for software implementation, and the coding efficiency is not acceptable for many real-time applications. To overcome the problems and further improve efficiency, several hardware oriented methods and architectures are proposed. First, an improved disparity matching method, referred to as two-level hybrid disparity estimation, is proposed; it is designed for stereoscopic video and has low computational complexity and good performance. Then, a new segmentation algorithm for the depth map previously obtained, referred to as average but keep boundary (ABKB), is proposed. Both of these algorithms are combined with an existing mean shift color segmentation algorithm to design a high speed video object extraction unit.