This paper proposes a high-speed object detection system based on a hierarchical parallel vision chip. In architecture, the vision chip architecture mainly contains three hierarchical processors: a pixel-parallel processing element (PE) array, a patch-parallel patch processing unit (PPU) array and a dual-core microprocessor (MPU). The three processors can perform low-level, mid-level and high-level image processing, respectively. In algorithms, the state-of-the-art feature operators: local binary pattern (LBP) and histogram of orientation gradient (HOG), and the efficient classifier AdaBoost are employed on the parallel vision chip. Experimental results demonstrate that the proposed system enhance the performance of object detection significantly. The object detection for a 128×128 resolution image can be achieved in less than 0.1ms on the proposed system.