The goal of neuromorphic computing is to understand brains better and thereby build better computers. In this paper, we describe a special-purpose hardware architecture for neural network simulation systems called neuron machine, and propose novel schemes that can be used effectively for large-scale neuromorphic simulations. A neuron machine system consists of a single digital hardware neuron, which is designed as a large-scale fine-grained pipelined circuit, and a memory unit called network unit. By using a large number of memories and extensive pipelining, we can enable a neuron machine system to exploit a large amount of the parallelism inherent in neural networks, while retaining the flexibilities of network topology. A multi-time-scale scheme enables synaptic and neuronal functions to be simulated with different time scales, and thereby considerably improving the hardware utilization. In addition, our multi-system scheme synchronously connects multiple neuron-machine systems to simulate larger-scale neural networks while retaining the speed of each machine. As an example of the proposed architecture, a simulation system for the networks of biologically realistic Hodgkin-Huxley neurons capable of complex synaptic features such as spike-timing dependent plasticity and dynamic synapse, is implemented on both a field-programmable gate array (FPGA) and a hardware simulator. Our system implemented on a single mid-range FPGA chip computed at a speedup of 1200x over a CPU-based system. The full source code of the hardware simulator written in MATLAB is available at a website and the readers can execute the code on the fly and reproduce the proposed schemes.