In many practical applications, it is difficult or impossible to obtain the exact solution of the mathematical model due to the limitations of solving methods and the complexity of the neural network itself. A natural problem is given as follows: does the exact solution of quaternion-valued neural networks (QVNNs) exist when successively improved approximate solutions can be obtained? Fortunately, the Hyers–Ulam stability happens to be one of the important means to deal with this problem. In this article, the issue of Hyers–Ulam stability of QVNNs with time-varying delays is addressed. First, inspired by the Hyers–Ulam stability of general functional equations, the concept of the Hyers–Ulam stability of QVNNs is proposed along with the QVNNs model. Then, by utilizing the successive approximation method, both delay-dependent and delay-independent Hyers–Ulam stability criteria are obtained to ensure the Hyers–Ulam stability of the QVNNs considered. Finally, a simulation example is given to verify the effectiveness of the derived results.