This article studies the time-varying output group formation (TVOGF) tracking problem for heterogeneous multi-agent systems (HMASs) with switching topologies. The primary objective is to design a TVOGF tracking protocol such that the followers are driven to form the desired sub-formations and track the corresponding output of the leader in each subgroup. Firstly, distributed observers are developed to estimate the states of leaders with switching topologies. A TVOGF tracking controller is then designed based on the proposed observers. It is shown that under the distributed protocol, the TVOGF tracking control problem for HMASs with switching topologies can be solved if the average dwell time associated with the switching topologies is larger than a fixed threshold. Finally, an example is provided to demonstrate the effectiveness of the proposed protocol.