Radio frequency (RF)-based drone identification is a safety-critical task, where erroneous model outputs may result in potential costs. However, most existing studies are based on the closed-set assumption, with only a few studies considering the presence of out-of-distribution (OOD) samples during testing. This paper argues that a reliable model should strive to minimize errors in classification, necessitating the ability to detect both OOD and misclassified ID samples (ID✗), as both types can lead to model misclassifications. We term this task Reliable Drone RF Signal Identification (RDI). To address the RDI task, this paper proposes an extreme value theory (EVT)-based method for simultaneously detecting these two types of samples. Initially, we propose employing a Generalized Pareto Distribution (GPD) model and establishing an uncertainty scoring function for each ID class, based on correctly identified in-distribution samples (ID✓). The proposed function effectively assigns lower uncertainty to ID✓ and higher uncertainty to ID✗ and OOD samples. Moreover, we tackle the GPD threshold selection issue, given its paramount significance in the construction of the GPD model. This paper proposes a GPD threshold selection algorithm based on minimized Kullback-Leibler (KL) divergence (MKL-based GPDTS). MKL-based GPDTS computes an appropriate GPD threshold for each ID class, based on the information of that class. We conduct comprehensive validation of the proposed approach using a self-collected drone RF signal dataset and an open-source dataset. The experimental results demonstrate the effectiveness of the proposed method.