This paper is concerned with multiple Lagrange stability under perturbation for recurrent neural networks with time-varying delays. This is different from traditional Lagrange stability, which means that multiple Lagrange stability under perturbation holds for any disturbance of initial value and any structural perturbation, within a specified and well-characterized set. In this paper, multiple Lagrange stability under perturbation with respect to a finite number of trivial solutions is established, which has good robustness. Under certain perturbation, the core of the proof of the boundedness of trajectories in mutually disjoint subregions is to employ the generalized differential inequality. The results supplement and extend some previous results, and they are applicable to robust analysis of multiple equilibria. Finally, numerical calculation and simulations are implemented to illustrate the effectiveness of the theoretical results.