Many of today's Visual-Inertial Odometry (VIO)frameworks work well under regular motion but have issues and need special treatment under special motion. Here, special does not imply bad or corrupted data but stands for increased difficulty to treat clean data. Common special motion for VIO are large feature displacement due to fast motion close to a scene and zero motion phases not providing sufficient baseline. In this paper we present a feature and frame selection approach which seamlessly handles all motion scenarios without the need of (error prone)motion case identification and subsequent case-specific heuristics. We further show that this approach allows to eliminate features in the state vector (persistent features)altogether while still being able to inherently handle zero motion phases. This reduces computational complexity while maintaining the ability to hover in place. We integrate our frame selection approach into our own VIO algorithm and compare its performance against three state-of-the-art algorithms with real data on a real platform. While our approach shows slightly higher global drift it is the only algorithm that can reliably estimate the pose over a large motion spectrum from fast scale change down to zero motion.