Federated edge learning (FEEL) is a novel paradigm that enables privacy-preserving and distributed machine learning on end devices. However, FEEL faces challenges from data/system heterogeneity among the participating clients and resource constraints of edge networks, which affect the efficiency and accuracy of the learning process. In this paper, we propose a comprehensive framework for client selection in FEEL based on the concept of Value-of-Information (VoI), which measures how valuable a client is for the global model aggregation. Our framework consists of two independent components: a VoI estimator that uses reinforcement learning to learn the relationship between VoI and various heterogeneous factors of clients; and a greedy client selector that chooses the most valuable clients under network resource constraints. Compared with most of the previous works that use concrete criteria to evaluate and select heterogeneous clients, our VoI-based approach is more comprehensive. Extensive experiments on different datasets and learning tasks are conducted, which show that our framework outperforms several state-of-the-art methods in terms of accuracy.