In this paper, a high-speed elliptic curve cryptography (ECC) processor specialized for primes recommended by the National Institute of Standards and Technology (NIST) was constructed. Toom-Cook multiplication without division was proposed to implement modular multiplication for NIST primes. Compared with a traditional algorithm, the computation complexity was reduced from 16 base multiplications to 7 in 4-way Toom-Cook multiplication. Moreover, we introduced non-least-positive (NLP) form into our design, so that the carry chain in the large array accumulation was broken down, which greatly shortened the critical path and made parallel processing possible. In order to support NLP form and lazy reduction strategy, conventional fast reduction methods for NIST primes were also modified. In addition, pipeline technique at the level of point multiplication was used, so the latency of modular inverse can be covered. Implemented on the Xilinx Virtex-6 FPGA platform, the ECC processor can perform a point multiplication every 54 μs at the cost of 30.3k LUTs and 48 DSPs. Synthesized with 180nm CMOS technology, the speed achieves 43.7 μs with 466k gate counts. These experimental results show a significantly better performance per area than previous works.