Updating the state of reservoir nodes is one of the essential operations of reservoir computing (RC), which highly affects the system’s performance. In an echo state network (ESN), one of the primary types of RC, the process of state renewal can be divided into two stages: multiplication of the weight matrix with the input-state vector and applying a nonlinear activation function on the sum of products. The weight matrix is typically large and sparse, providing opportunities for optimizing the matrix multiplication; the choices of activation functions may also affect hardware resource utilization. This paper introduces an optimized reservoir node architecture for FPGA-based RC systems. Specifically, we adopt the bit-serial matrix multiplier and direct spatial implementation of the weight matrix to fully exploit the sparseness property. The canonical signed digit representation is also employed to further optimize the multiplier logic. Furthermore, a hyperbolic tangent activation function is designed and optimized to maintain the nonlinearity of the neural network without affecting its accuracy. Compared with existing hardware ESN designs, our reservoir node architecture significantly reduces resource utilization while maintaining comparable performance.