Collecting relevant and high-quality data is critical to machine-learning-based application development in automotive industry. It is highly desired to concentrate the connected vehicle data collection efforts on a set of useful data to avoid irrelevant or redundant data. For a given machine learning (ML) task, without a priori information, it can be challenging to determine what features to collect. In this work, we propose an efficient data collection workflow featuring a loop involving a connected vehicle fleet, a ML algorithm and a feedback-based dynamic feature selection strategy. The feature selection process is embedded in the data collection workflow, therefore it is more efficient than the traditional workflow where feature selection is applied after all data are collected. The proposed workflow reduces data transmission and storage cost. A feature selection algorithm that exploits and explores feature sets using an upper-confidence-bound-based method is designed for the trade-off between collecting known useful data and collecting potentially useful data. Validations are performed on two classification tasks to mimic the connected vehicle data collection: a synthetic problem based on a public ML dataset, and a robot ground surface classification problem with experiment data in a simulated data collection environment. The proposed method is tested against several baseline methods, including collecting all available data before feature selection, collecting random features, and running the data collection procedure episodically with embedded feature selection. Results show that the proposed algorithm in the closed-loop procedure achieves adequate performance with relatively few irrelevant and redundant data.