As the qubit capacity of current quantum computers is insufficient for many real-world machine learning problems that require the processing of a large number of features, hybrid methods are often used as an alternative for purely quantum models. This includes quantum transfer learning, a hybrid technique that can be applied to a variety of tasks, such as classifying large images. However, as this approach is hybrid in nature, it is not always evident what part of the algorithm is ultimately responsible for the performance. More specifically, while a hybrid method like quantum transfer learning may deliver good results, it is crucial to examine to what extent the quantum part contributed to the overall performance, as this often remains elusive. In this work, we investigate the quantum impact in a hybrid classical-quantum transfer learning approach. We run multiple experiments in various scenarios and show that the impact of the quantum part is, in fact, only minuscule and highly dependent on the classical part of the approach. Our results furthermore indicate that quantum transfer learning does not necessarily provide a significant advantage or improvement over a regular variational quantum circuit approach when the classical part is reduced to a mere feature extractor, and no further classical layers are added to be trained simultaneously to the quantum part.