In this brief, the optimal containment control problem for a class of unknown nonlinear multi-agent systems (MASs) is studied via a time-aggregated (TA) model-free reinforcement learning (RL) algorithm. First, based on the idea of TA, the control policy is updated only when the system visits a finite subset of the state space. Thus, the control is event-triggered and not time-triggered. On this basis, a model-free TA-based value iteration (TA-VI) algorithm is proposed to learn the optimal control protocol. Since the finite important states are considered and the control is event-triggered, this algorithm requires fewer updating times and fewer computation than the conventional optimal containment control. Moreover, the TA-VI algorithm eliminates requirements on the function approximator and state discretization, which allows a strict convergence analysis via the mathematical induction method. Finally, simulation results are given to show the feasibility and superiority of the proposed algorithm.