Federated learning (FL) enables devices to update a global model while keeping the training data local, so that data privacy is protected. However, the local data of devices is usually non-independent and identically distributed (non-i.i.d.), which leads to performance degradation. This paper aims to address this issue by a client-selection approach. In particular, in consideration of balancing the label distribution of the selected clients, a new client selection method called grouping based scheduling (GS) scheme is proposed, with which clients are divided into several groups based on a new metric called group earth mover’s distance (GEMD). Experiment results show that the GS can improve the performance of FL algorithms, compared to the random scheduling scheme. An encryption method is further proposed to enhance privacy protection, which facilitates the application of the proposed GS scheme.