Deep Learning methods are widely applied in Robotics and Automated Driving scenarios. The task of perception for Automated Driving in the real world is particularly challenging and requires a sufficient amount of high quality labeled training data for the algorithms to perform well. However, the means of obtaining real world datasets are limited. It is common practice to have human labelers involved at least to some extent. Regardless of whether the process is partially automated or not, these labels never represent perfectly accurate ground-truth. By investigating the recognition performance of a state-of-the-art object detector as a function of the quality of a labeled real world training set, we study the effect of labeling errors of various types and severity. To this end, the given labels are treated as a reference to which synthetic errors are added systematically in order to determine the performance of the object detector if trained on the erroneous dataset.