This paper describes fast implementations of Efficient Second-order Minimization method for region-based visual tracking of a planar target on a multi-core CPU with SIMD instructions, without relying on GPUs. We show that careful implementation considerations lead to high performance even only with scalar instructions on a single core, and full utilization of the parallelism achieves a frame rate of more than 3500 fps for a template with 64 × 64 pixels and 1200 fps for 128 × 128 pixels on Intel Core i7-4600U processor. Real-time visual tracking experiments with 450-fps USB 3.0 shows its effectiveness in high-frame-rate tracking applications.