Controlling large-scale dynamic population systems, known as ensemble control, is a pervasive and essential task in many emerging applications from diverse scientific domains. Previous focuses in the area of ensemble control have been placed on seeking open-loop control strategies due to the unavailability of state feedback information for each individual system in the ensemble. In this article, we develop a foundational framework for analysis and control of ensemble systems with closed feedback control loops. We introduce the notion of ensemble moments and construct moment systems associated with the ensemble systems. By extending the classical moment problem in mathematical analysis and statistics, we establish duality between ensemble systems and their moment systems and reveal the dynamic equivalence, e.g., controllability, between these dual systems.