In this paper, we propose a federated extreme learning machine system (Fed-ELMS) to meet the demand for federated learning scenarios. In the scenario of federated learning, data is kept on edge devices to preserve the privacy of data, while metadata, such as model parameters, are exchanged between a centralized cloud server and edge devices. Despite non-independent and identically distributed (non-IID) and imbalanced data across edge devices, we show that Fed-ELMS can still achieve comparable performance with only 3.3% accuracy loss compared to a centralized ELM trained with IID and balanced data. (a) Moreover, by quantizing input weights and biases, parameters of a model and transmission power consumption between cloud and edge are dramatically reduced. Compared With conventional neural networks (NNs) with the same transmission cost, the proposed Fed-ELMS outperforms FederatedAveraging NN (Fed- NN) by 2.3% accuracy and the fine-tuning process is 7%-33% less time-consuming. Therefore, the proposed Fed-ELMS is a promising system for edge devices to support the future trend of federated learning.