This article tries to solve the challenges in few-shot transfer learning for anomaly detection: how to guarantee the transfer effect on insufficient even limited source domain data, and how to make the transfer process interpretable for getting trustworthy results. This article proposes a deep domain-adversarial contrastive model with time-frequency transferability analytics. The essential idea is extracting fine-grained information from different frequency bands for reliable transfer. First, a time-frequency domain feature pool is constructed by applying wavelet scattering network (WSN) under different decomposition scales and rotation orientations. An orientation-first selection strategy is further designed to determine the optimal features that can cover the low, medium, and high frequency bands. A new transferability metric, named frequency importance metric (FIM), is then built through frequency hypersphere matching to quantify the significance of each frequency band from source domain data. Second, a deep domain-adversarial contrastive network (DDCN) is constructed to realize selective information transfer according to frequency band’s significance. In DDCN, a purposeful feature representation can be extracted through the contrastive learning between the deep features and wavelet features that are weighted by FIM, thus leading to valid transfer in few-shot scenario via domain-adversarial training. Experiments are conducted on the two typical anomaly detection problems, i.e., image recognition detection on the MNIST~USPS and Office-Home data sets, and early fault detection on the IEEE prognostic and health management Challenge 2012 bearing data set. The results not only verify the superior performance of the proposed method to the few-shot transfer learning but also reveal the frequency saliency in the transfer process.