Since its outbreak reported in late 2019 in Wuhan, China, the novel coronavirus disease (COVID-19) has been the major challenge across the globe, affecting virtually all aspects of our lives. To effectively manage the pandemic, we need fast, non-invasive, and precise routines for detecting active COVID-19 cases. Although there exist deep learning approaches for detecting COVID-19 in medical image data, their generalization abilities remain unknown. We tackle this issue and introduce deep ensembles that benefit from a wide range of architectural advances, alongside a new fusing approach to deliver accurate predictions. Also, we evolve their content to not only accelerate the inference but also to boost the classification performance. Our experiments, performed on a number of datasets of chest X-ray images, show that the proposed technique renders high-quality classification and generalizes well over a variety of test scans.