Data auditing is a process to consistently keep the quality of data high, but this process is generally missing in network security monitoring. When network-based intrusion detection systems catch any suspicious packet, they generate alert messages that are further investigated by security analysts. An alert is generally assigned to only one analyst at best, and then the analyst determines whether the alert is true or false, called labeling. Therefore, different analysts may label very similar alerts with different labels. In this article, we introduce this problem of inconsistent labeling in network security monitoring and present a new automatic data auditing method to check if any human mistake has occurred for the labeling. Through our experiments on two data sets, a private one from a real security operations center and an open data set for reproducible experiments, we confirm that the new auditing method can catch incorrect labels, and the accuracy of a machine learning model on the data set can be enhanced through the label correction.